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PREFACE 


This  report  was  prepared  at  Veda  Incorporated  in  Dayton,  Ohio.  The  work  was  performed  during 
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Major  Julie  Cohen  served  as  the  Program  Manager  and  Mr.  Philip  V.  Kulwicki  served  as  the 
Project  Engineer.  Mr.  Michael  Rountree  was  the  Veda  Program  Manager. 

The  industry  review  was  conducted  by  Major  Julie  Cohen  and  Mr.  Philip  Kulwicki  of  the  Crew 
Systems  Directorate,  Armstrong  Laboratory;  Mr.  Michael  Rountree  and  Mr.  Edward  Lehman  of 
Veda  Incorporated;  and  Mr.  Brett  Storey  of  Storey  Consulting.  This  report  was  prepared  by  Mr. 
Edward  Lehman,  Mr.  Michael  Rountree,  Mr.  Brett  Storey,  Mr.  Philip  Kulwicki,  and  Major  Julie 
Cohen,  with  the  editorial  assistance  of  Ms.  Katherine  Jackson. 
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INTRODUCTION 


The  Air  Force’s  Crew-Centered  Cockpit  Design  (CCCD)  Program  is  an  advanced  development 
effort  to  upgrade,  demonstrate,  and  evaluate  a  crew  station  design  process,  known  as  the  Crew- 
Centered  System  Design  Process  (CSDP),  and  its  support  procedures,  databases,  and  tools.  Past 
and  present  crew  station  designs  were  primarily  driven  by  the  configuration  of  the  aircraft,  and 
little  attention  was  given  to  operability  by  the  crew.  The  CSDP  is  intended  to  improve  design 
practice,  allowing  designers  to  base  decisions  on  mission  requirements  and  crew  capabilities  while 
meeting  installation  constraints.  Once  it  is  validated,  the  CSDP  should  lead  to  the  development  of 
well-integrated,  intuitive  crew  stations.  A  key  part  of  this  development  is  a  computer  design 
support  system,  known  as  the  Cockpit  Design  System  (CDS),  which  has  a  software  and 
simulation  toolset  to  fully  support  the  cockpit  developer’s  need  for  effective  crew  station  analysis, 
design,  and  test.  The  CDS  Toolset  offers  improved  design  efficiency  and  includes  traceability 
functions  that  preserve  the  rationale  for  design  decisions  (e.g.,  automation  alternatives,  location  of 
controls  and  displays,  selection  of  display  symbols)  to  guide  crew  station  upgrades.  The  users  are 
cockpit  design  and  test  teams  in  government  and  industry.  The  goals  of  the  program  are  to 
structure  and  manage  the  total  crew  system  development,  provide  validated  tools  for  cockpit  design 
and  test,  and  ensure  that  crew  capabilities  and  limitations  are  prominent  design  variables. 

The  CCCD  Program  Office  recognizes  that  the  acceptance  and  long-term  utility  of  the  CCCD 
products  depend  on  the  interest  and  support  of  the  cockpit  development  community,  which 
includes  aircraft  prime  contractors  (also  called  industry)  and  government  aircraft  acquisition 
organizations.  For  this  reason,  a  survey  of  four  aircraft  prime  contractors  was  made  during 
August  and  September  of  1993.  The  companies  surveyed  were:  Northrop  Advanced  Technology 
Development  Center,  Lockheed  Fort  Worth  Division,  McDonnell  Douglas  Aerospace-East,  and 
Lockheed  Aeronautical  Systems  Company.  This  report  documents  the  results  of  the  survey. 

The  objective  of  the  survey  was  to  elicit  end-user  requirements  for  CCCD  products,  where  an  end- 
user  is  defined  as  any  member  of  a  crew  system  organization  that  will  have  a  direct  impact  on  the 
outcome  of  a  cockpit  design.  The  two  principal  organizations  that  are  responsible  for  cockpit 
development  are  the  aircraft  manufacturers  crew  system  groups  and  the  System  Program  Office 
(SPO)  crew  system  groups.  An  aircraft  crew  system  group  directly  performs  the  analysis,  design, 
and  evaluation  of  a  cockpit,  while  a  SPO  group  helps  to  structure  aircraft  user  (mission-specific) 
requirements  and  monitors  development  programs  so  that  the  cockpit  will  meet  or  exceed 
requirements.  The  two  groups  share  several  key  characteristics.  Both  need  to  work  with  other 
design  teams  involved  with  air  vehicle  development.  Both  are  responsible  to  similar  higher 
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organizations  and  produce  specific  products.  The  intent  of  the  CCCD  Program  is  to  provide  a 
process  and  a  toolset  to  assist  in  performing  the  necessary  design  activities  with  greater  efficiency 
and  increased  success. 

The  survey  was  performed  in  two  parts.  First,  by  advance  mailing,  the  industry  review  teams 
(reviewers)  were  given  an  Industry  Review  Package  that  contained  a  narrative  description  of  the 
new,  structured,  traceable,  and  crew-centered  process  for  cockpit  design  (Appendix)  and  a 
narrative  scenario  that  provided  additional  context.  After  examining  the  narrative  description  and 
scenario,  the  reviewers  critiqued  each  document  by  completing  a  questionnaire.  Details  of  the 
entire  survey  can  be  found  in  the  CCCD  Interim  Technical  Report  No.  1  (Reference  1).  Second, 
after  reviewing  the  completed  questionnaires,  an  Air  Force  team  visited  each  company  to  conduct 
follow-up  discussions  and  to  clarify  the  critiques.  Following  the  Air  Force  visit,  each  company 
supplied  a  written  final  report. 

The  reviewers  had  varied  experience,  composition,  and  respective  roles  in  the  cockpit  development 
process.  A  wide  range  of  experience  in  differing  phases  of  aircraft  development  was  captured 
through  various  team  experiences  in  Concept  Exploration,  Demonstration/Validation,  Engineering 
and  Manufacturing  Development,  and  Post-Production  Support. 

Information  was  obtained  from  engineers,  scientists,  and  managers  who  were  involved  in  different 
crew  station  activities  including  pilot-vehicle  interface  (PVI)  design,  crew  station  design,  human 
factors  analysis,  systems  engineering,  and  operational  analysis.  These  disciplines  are 
representative  of  those  found  in  an  integrated  crew  system  design  group.  Most  crew  system 
design  groups  in  the  aircraft  industry  appear  to  have  formal  or  informal  ties  with  the  other 
disciplines,  essentially  forming  Integrated  Product  Team  (IPT)-like  organizations.  Only  one 
company  had  formally  used  the  IPT  methodology  in  the  crew  system  design  area,  but  most  have 
working  groups  who  provide  similar  results.  One  of  the  strongest  indications  of  the  IPT-like 
organizational  philosophy  was  a  common  recognition  by  survey  participants  that  crew  system 
groups  must  interface  with  the  various  disciplines  that  are  responsible  for  the  other  aircraft  systems 
affected  by  the  cockpit. 

The  format  of  each  visit  was  as  follows:  The  Air  Force  team  provided  opening  remarks  and 
reviewed  the  agenda,  after  which  all  attendees  introduced  themselves.  The  agenda  included 
summaries  of  the  objectives  and  background  of  the  CCCD  Program;  the  evolution  of  the  CSDP 
and  the  CDS  Toolset;  and  the  objectives,  status,  and  background  of  the  ongoing  CCCD  Field 
Demonstration  Program.  The  Air  Force  team  then  conducted  a  question-by-question  review  of  the 
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written  survey  responses.  Finally,  the  group  discussed  typical  design  facility  practices,  the  data 
processing  environment,  and  prioritization  of  the  CSDP  and  CDS  technical  objectives.  Additional 
time  was  devoted  to  demonstrations  of  resident  cockpit  design  and  evaluation  tools.  The  visits 
concluded  with  a  brief  discussion  of  major  conclusions  and  plans  for  future  interaction.  The 
participation  in  this  survey  is  summarized  in  Table  1. 


Table  1.  Survey  Participants* 


Date 

Number 

Aircraft 

Number 

Aircraft 

Company 

of 

of 

Dev 

of 

Dev 

Visit 

Reviewers 

Exp 

Attendees 

Exp 

Northrop  Advanced 
Technology  Development 
Center 

Aug  93 

5 

B-2 

A/FX 

5 

B-2 

A/FX 

Pico  Rivera,  CA 

F-16 

F-16 

Lockheed/Fort  Worth 

Sep  93 

F-22 

F-22 

Fort  Worth,  TX 

l 

A-12 

l 

A-12 

A/FX 

A/FX 

McDonnell  Douglas 

Aerospace  -  East 
(MDA-E) 

St  Louis,  MO 

Sep  93 

2 

YF-23 

F-18 

F-15 

2 

YF-23 

F-18 

F-15 

F-22 

Lockheed  Aeronautical 

A/FX 

JPATS 

P-7 

Systems  Company  (LASC) 

Sep  93 

2 

5 

C-130 

Marietta,  GA 

A/FX 

JPATS 

*  Reviewers  of  the  CSDP  narratives,  who  supplied  the  detailed  critique  and  questionnaire  results,  did  not  correspond 
one-to-one  with  the  attendees  at  the  follow-up  visit. 


Overall,  the  participants  agreed  that  the  CCCD  Program  was  making  progress.  Each  company 
representative  made  specific  suggestions  for  enhancements  and  emphasized  features  that  could  be 
valuable  to  both  industry  and  government.  This  cooperative  approach  will  contribute  greatly  to  the 
development  of  a  process  and  support  tools  that  will  improve  future  cockpit  development  projects. 
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COMMENTS  ON  THE  CSDP 


The  data  collected  from  the  written  responses  and  from  the  technical  discussions  at  each  company, 
along  with  the  interviewer’s  interpretations,  are  presented  in  the  following  paragraphs.  In  many 
cases,  the  responses  represent  a  consensus  of  opinion,  although  some  are  individual  opinions. 

General  Comments  on  the  CSDP 

Overall  Need.  The  reviewers  generally  concurred  that  there  is  value  in  having  a  uniform,  accepted 
process  for  crew  station  development.  This  kind  of  process  would  foster  the  use  of  effective 
techniques  and  provide  instructional  material  for  newer  project  managers.  A  process  description 
with  more  depth  could  also  be  valuable  to  the  members  of  the  Cockpit  Design  Team  (design  team) 
who  are  assigned  to  the  development  or  modification  of  a  cockpit. 

Target  Audience.  Some  reviewers  pondered  the  overall  purpose  of  the  CSDP  in  the  following 
manner:  If  the  target  audience  consists  of  novice  users,  they  may  not  have  the  requisite  experience 
to  effectively  apply  the  CSDP.  In  other  words,  they  may  know  the  “whats,”  but  not  the  “hows”  of 
the  specific  procedures  (see  next  paragraph).  If  the  CSDP  is  developed  for  experienced  users,  they 
would  know  how  they  have  developed  cockpits  in  the  past,  and  only  the  CDS  Toolset  would  be 
useful.  If  the  CSDP  was  meant  to  ensure  that  industry  competitors  have  a  “level  playing  field,”  it 
was  deemed  to  be  a  worthwhile  accomplishment.  The  conclusion  is  that  the  description  should  be 
rewritten  to  clarify  the  intended  role  of  the  CSDP. 

Level  of  Detail.  As  anticipated,  the  reviewers  requested  more  in-depth  information  about  the 
CSDP  activities.  However,  this  additional  information  will  be  provided  on  an  on-demand  basis 
because  the  brevity  of  the  existing  CSDP  was  judged  to  be  appropriate  for  both  management  users 
and  experienced  designers. 

Procedure  Outline.  In  the  CSDP,  each  activity  in  the  design  process  is  accomplished  by  following 
a  detailed  procedure  for  performing  the  work.  A  sample  procedure  was  included  in  the  advance 
mailing  and  was  reviewed  during  company  visits.  No  additions,  deletions,  or  modifications  to  the 
procedures  were  suggested. 

Tailorabilitv.  The  reviewers  concurred  with  the  objective  to  make  the  process  tailorable  to  the  time 
and  resource  limitations  of  an  actual  project.  Several  reviewers  expressed  support  for  tailorability 
in  the  current  CSDP,  asserting  that  it  should  provide  a  basic  approach  to  cockpit  development  that 
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can  be  adapted  to  meet  the  needs  of  a  given  program.  They  also  commented  that  the  CSDP  makes 
the  human-centered  emphasis  more  explicit  and  goes  beyond  other  system  process  descriptions  by 
emphasizing  the  iterative  nature  of  the  design  approach. 

Phasing.  Several  comments  referred  to  the  need  to  segment  the  CSDP  by  program  phases, 
because  the  nature  and  depth  of  CSDP  activities  are  heavily  influenced  by  the  maturity  of  the 
development  program.  The  Weapon  System  Development  Program  (WSDP)  phases-Concept 
Exploration,  Demonstration/Validation,  Engineering  and  Manufacturing  Development-were 
discussed  as  one  possible  framework  for  phasing.  Another  possible  framework  that  was 
discussed  is  based  on  different  project  categories.  Three  project  categories  were  defined:  (1)  a 
minor  change  to  an  existing  cockpit,  such  as  an  Engineering  Change  Proposal  (ECP);  (2)  a  major 
block  upgrade  of  an  existing  cockpit;  and  (3)  a  ‘otally  new  cockpit  development  effort.  Regardless 
of  the  framework,  it  was  agreed  that  the  activities  that  are  conducted  at  each  of  these  levels  tend  to 
be  similar,  but  the  depth  of  the  analysis  (or  design  or  test)  and  the  proportion  of  time  spent  doing 
them  varies  greatly. 

Air  Force  Endorsement.  The  reviewers  urged  the  Air  Force  to  incorporate  the  CSDP  into  Requests 
for  Proposal  (RFPs),  Statements  of  Work  (SOWs),  and  other  contractual  agreements.  This  will 
give  the  design  team  the  necessary  authority  and  justification  to  acquire  the  tools  and  funding 
resources  needed  to  accomplish  the  process.  As  one  reviewer  stated,  “Any  time  the  Air  Force  can 
provide  guidance  and  tools,  this  is  desirable.” 

Level  in  Work  Breakdown  Structure  (WBS).  One  reviewer  described  the  crew  system  as  “the 
poor  stepchild”  with  respect  to  most  other  air  vehicle  subsystems.  Several  discussions  at  each 
company  focused  on  a  previous  Air  Force  initiative  to  raise  the  crew  system  to  a  Level-3  WBS  item 
under  MIL-STD-881B,  with  the  recommendation  (at  each  site  visit)  that  the  Air  Force  pursue  this 
initiative.  While  this  would  not  appear  to  directly  impact  the  development  of  the  CSDP,  it  would 
elevate  the  importance  of  using  a  process  like  the  CSDP  within  the  overall  air  vehicle  program, 
thereby  enhancing  the  probability  of  its  acceptance  and  use  by  industry. 

Integrated  Product  Team  (IPT).  Many  reviewers  cited  the  need  to  relate  the  CSDP  to  the  entire 
aircraft  project.  These  ties  should  include  the  other  engineering  disciplines  such  as  avionics, 
propulsion,  and  flight  control.  The  CSDP  is  fully  compatible  with  the  IPT  concept,  in  which 
Human  Factors  Engineering  (HFE)  is  integral  to  the  team. 


5 


It  was  pointed  out  that  critical  cockpit  development  milestones  should  be  applied  at  the  program 
level  and  integrated  with  all  other  critical  milestones,  to  ensure  that  cockpit  requirements  and  issues 
are  addressed  appropriately.  Several  of  the  reviewers  were  acquainted  with  the  Air  Force  Crew 
System,  System  Engineering  Master  Schedule  (SEMS),  and  mentioned  the  probable  value  of 
correlating  the  CSDP  with  the  SEMS.  The  acquisition  SPOs  would  use  the  SEMS  to  manage  the 
crew  system  progress  on  major  development  contracts. 

Extended  Parallelism  of  Activities.  Most  reviewers  agreed  that  the  CSDP  activities  should  run 
more  concurrently  throughout  the  developmental  life  cycle  than  was  implied  by  the  survey 
materials.  For  example,  it  was  stated  that  part-task  simulation  tends  to  begin  earlier  than  implied  in 
the  present  CSDP,  and  that  planning  activities  tend  to  continue  further  into  the  process  than 
graphically  shown.  The  concurrence  of  the  activities  will  be  modified  in  future  CSDP  revisions. 
However,  the  sequential  conduct  of  analysis  prior  to  design  will  remain  intact  to  enable  the 
traceability  from  derived  requirements  to  implementation  decisions. 

Interaction  and  Iteration  of  Activities.  There  was  widespread  agreement  that  the  CSDP  would 
benefit  both  from  a  more  explicit  interaction  among  various  crew  system  development  activities  and 
from  a  more  explicit  description  of  the  iterations  through  the  activities.  A  more  detailed  description 
of  the  application  of  the  CDS  Toolset  throughout  the  process  was  also  recommended. 

Some  reviewers  remarked  that  the  CSDP  should  provide  guidance  in  determining  the  number  of 
iterations  of  the  analysis/design/evaluation  cycle.  Other  reviewers  observed  that  iterations  are 
continued  within  time  and  budget  constraints,  as  now  indicated  in  the  CSDP.  The  general 
consensus  was  that  this  practical  approach  is  essentially  the  way  the  iterations  are  determined. 
Another  suggestion  was  that  entry  and  exit  criteria  could  be  addressed  in  the  CSDP. 

Assistance  in  Scoping  Work.  At  one  company,  reviewers  suggested  that  the  CSDP  should  provide 
a  basis  for  estimating  the  scope  of  work  within  the  phases  and  activities.  This  estimate  would  be 
helpful  for  companies  or  organizations  that  have  not  had  extensive  experience  in  cockpit 
development.  However,  the  majority  of  the  reviewers  felt  that  labor-hour,  time,  and  cost  data  are 
not  tracked  in  a  manner  that  would  permit  the  allocation  to  CSDP  activities  or  functions. 
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Specific  Comments  on  the  CSDP  -  Phasing  of  the  Process 


The  consensus  among  reviewers  was  that  the  CSDP  is  properly  portrayed  in  the  five  distinct,  but 
interactive  categories  of:  (1)  Program  Planning,  (2)  Up-Front  Analysis,  (3)  Crew  System 
Analysis,  (4)  Crew  System  Design,  and  (5)  Crew  System  Evaluation.  By  distinguishing  five 
distinct  categories,  each  type  of  activity  can  be  coherently  described  and  developed. 

Program  Planning 

The  description  of  the  Program  Planning  category,  which  includes  scheduling,  was  the  smallest 
section  of  the  draft  CSDP  document  because  of  the  focus  on  the  technical  needs  of  the  design  team. 
The  reviewers  observed  that  there  are  a  few  key  needs  that,  if  satisfied,  would  make  the  planning 
and  scheduling  of  an  integrated  cockpit  development  effort  more  efficient. 

Dynamic  Planning  Capability.  The  reviewers  would  like  to  be  able  to  perform  dynamic  planning  to 
accommodate  frequently  changing  requirements.  They  would  like  to  be  able  to  project  the  impact 
of  certain  situations,  both  real  and  postulated  and  to  replan  both  known  and  projected  situations  in 
order  to  react  to  ever-changing  program  directions.  In  fact,  this  capability  may,  in  the  long  run, 
actually  reduce  the  frequency  of  such  changes  because  users  will  have  the  tools  to  predict  the 
impacts  of  changes  while  they  are  still  in  the  early,  formative  stages  of  the  development  phase. 
The  CDS  currently  lacks  this  capability. 

Integration  with  Total  Program.  When  planning  and  scheduling  functions  are  performed,  the  crew 
systems  group  often  lacks  the  necessary  information  to  plan  a  complete  integration.  A  substantial 
amount  of  time  is  required  to  track  down  all  of  the  information  and  to  keep  it  updated  to  reflect 
changes.  Without  this  information,  cockpit  designers  may  not  complete  certain  tasks,  or  may 
perform  them  in  an  unplanned  sequence,  which  could  preclude  an  orderly  integration  with  other 
subsystems.  Conversely,  other  design  groups  often  cannot  provide  the  crew  system  group  with 
the  needed  information  on  a  timely  basis.  Many  reviewers  stated  that  integration  at  all  levels, 
specifically  technically,  would  be  of  great  value,  particularly  given  the  growing  use  of  the  IPT 
structure.  The  CSDP  is  being  revised  to  make  interactions  with  other  program  elements  more 
explicit. 

Subcontractors  and  Risk/Technology  Gaps.  The  reviewers  felt  that  the  CSDP  should  be  set  up  to 
take  into  account  subcontractor  plans  and  schedules  because  they  are  critical  to  the  integration  of 
the  cockpit.  Additionally,  the  need  to  assess  the  risks  associated  with  both  current  technology  and 
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projected  technological  gaps  must  be  introduced  into  the  planning  function  of  the  cockpit 
development  process.  This  omission  was  an  oversight  in  the  draft  CSDP.  Assessment  of  risks 
will  be  an  integral  part  of  the  planning  phase  and  will  be  essential  in  the  technical  areas  of  the 
CSDP  as  well. 

Upwardly  Compatible  to  Project.  To  enable  a  fully  integrated  planning  and  scheduling  capability, 
the  reviewers  felt  that  any  functionality  at  the  crew  system  group  level  should  be  compatible  with 
the  company’s  project  planning  and  reporting  system.  Typically,  a  project-level  planning  system  is 
large,  is  hosted  on  a  computer  system  that  is  not  directly  accessible  to  the  crew  system  group,  and 
does  not  easily  permit  data  to  be  imported  or  exported.  Personnel  must  enter  the  planning 
information  into  their  own  systems,  and  provide  hard  copy  submittals  for  manual  entry  into  the 
project-level  systems.  This  method  usually  results  in  lost  data,  de-emphasis  of  the  cockpit  portions 
of  the  project  plans,  and  duplication  of  effort.  Ideally,  the  CSDP  would  eliminate  or  minimize  this 
problem  with  its  explicit  program  planning  function  that  is  supported  by  CDS  software  tools.  The 
CDS  currently  lacks  this  capability. 

Crew  System  Plans.  One  of  the  elements  of  program  planning  that  was  of  marked  interest  to  the 
reviewers  was  how  the  CSDP  assists  in  the  preparation  of  crew  system  plans.  This  preparation 
entails  retrieving  and  using  information  that  shows  how  to  formulate  necessary  plans  based  on 
standard,  endorsed  formats,  and  how  to  apply  relevant  information  from  earlier  programs. 
Program  planning  has  grown  more  complex  because  of  the  IPT  function  and  because  of  additional, 
new  reporting  requirements.  The  CSDP  could  provide  useful  guidance  in  the  preparation  of  crew 
system  plans. 

More  Description/Activities.  One  of  the  requested  modifications  to  the  Program  Planning  category 
was  to  add  more  activities  that  would  provide  a  greater  amount  of  program  planning  and 
scheduling  functions.  Additionally,  the  descriptions  of  current  activities  need  to  be  developed 
before  an  accurate  review  of  this  category  can  take  place. 

Up-Front  Analysis 

The  majority  of  the  reviewers  were  pleased  with  the  formal  creation  of  a  separate  process  category 
for  requirement  definition.  Most  comments  on  this  category  were  positive.  Several  reviewers 
stated  that  this  portion  of  the  CSDP  provided  assistance  to  the  already-formulated  process  devel¬ 
opment  at  their  site. 
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New  Function.  The  formal  introduction  of  Up-Front  Analysis  to  the  development  of  the  cockpit 
was  recognized  as  an  appropriate  addition  to  the  development  cycle.  Up-Front  Analysis  activities 
are  usually  accomplished,  but  are  not  afforded  the  structure  implied  in  the  CSDP.  Most  reviewers 
stated  that  giving  more  attention  to  requirements  generation,  development,  and  traceability  at  an 
early  stage  will  be  of  benefit  in  subsequent  stages  because  a  more  definitive  set  of  requirements  can 
be  established  and  the  design  features  can  be  clearly  substantiated. 

Name  Change.  Several  reviewers  suggested  renaming  the  Up-Front  Analysis  category  to 
Requirements  Definition  because  the  latter  term  is  more  descriptive  and  conveys  more  weight  with 
program  planning  personnel.  In  view  of  the  broad  and  variable  interpretatio?  ie  word 
“requirements,”  this  part  of  the  CSDP  was  later  renamed  Requirements  Analysis  aru  redesign. 
This  new  title  is  used  throughout  the  remainder  of  this  document  to  refer  to  the  category  formerly 
known  as  Up-Front  Analysis. 

Interdisciplinary  Trade-offs.  One  of  the  main  themes  that  evolved  as  a  result  of  the  industry  s  -vey 
was  that  more  attention  should  be  paid  to  working  group  relationships  between  the  disciplines 
involved  in  developing  the  cockpit  and  its  associated  systems.  The  CSDP  took  into  account  a 
formal  method  of  multidisciplinary  systems  engineering,  that  of  an  integrated  design  team  that 
includes  engineers  from  a  number  of  disciplines.  While  this  approach  is  consistent  with  the  IPT 
approach  advocated  by  the  Air  Force,  it  apparently  does  not  reflect  the  prevailing  organizational 
structure  currently  in  use  in  industry  today.  Therefore,  multidisciplined  working  group  sessions 
should  be  shown  as  a  requirement  in  the  CSDP.  The  emphasis  during  the  Requirements  Analysis 
and  Predesign  phase  will  be  the  interpretation  and  trade-off  of  the  requirements  that  drive  the 
design  of  the  cockpit  and  its  associated  subsystems,  leading  to  the  development  of  integrated 
specifications  for  these  items. 

Accommodation  of  Requirements  Changes.  The  reviewers  characterized  cockpit  requirements  as 
highly  volatile  and  ever-changing,  and  emphasized  the  need  to  accommodate  these  changes  in  the 
CSDP.  While  not  specifically  discussed  in  the  Industry  Review  Package,  requirement  changes 
would  be  assessed  when  changes  occur  and  the  best  alternative  would  be  selected  to  meet  the 
modified  requirement.  For  example,  if  the  requirement  change  was  to  add  or  subtract  a  capability 
that  directly  affects  the  aircrew’s  ability  to  perform  a  mission,  then  the  analysis,  design,  and 
evaluations  performed  to-date  may  have  to  be  repeated.  In  contrast,  if  the  requirement  change  was 
to  extend  or  degrade  a  capability,  then  only  certain  design  elements  would  have  to  be  examined  for 
possible  impacts.  As  a  minimum,  an  impact  assessment  would  be  required,  whether  or  not  the 
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cockpit  design  needed  to  be  changed.  This  capability  will  have  to  be  more  formally  introduced 
throughout  the  CSDP  in  order  to  accommodate  changes  in  requirements. 


Crew  System  Analysis 

Discussions  of  Crew  System  Analysis  as  a  special  CSDP  category  generated  a  wide  range  of 
viewpoints.  Most  of  the  reviewers  remarked  that  the  CSDP  activities  and  sequences  would  be  a 
good  method  for  performing  these  analyses,  but  there  was  considerable  divergence  regarding  the 
value  of  the  analyses  versus  the  time  and  effort  expended. 

Activity  Development.  The  reviewers  considered  the  analysis  activities  described  in  the  Industry 
Review  Package  to  be  comprehensive  with  respect  to  the  design  of  the  PVI.  However,  the 
reviewers  expressed  a  concern  that  the  CSDP  did  not  include  enough  analysis  of  other  factors, 
such  as  life  support,  escape,  reach,  cockpit  layout,  canopy  and  structural  analysis,  ray  tracing,  and 
lighting  analysis. 

Concerns  Over  Time  and  Budget.  In  this  phase  more  than  the  others,  the  reviewers  were 
concerned  with  what  appeared  to  be  a  drastic  increase  in  the  number  of  activities  over  those 
currently  required  and  the  additional  time  necessary  to  perform  and  document  them.  Many  stated 
that  they  would  never  have  the  budget  to  support  these  activities  in  addition  to  the  other  more 
important  design  and  test  activities.  The  reviewers  expressed  a  desire  to  gain  experience  with  the 
CSDP  and  the  CDS  Toolset  before  commenting  further  on  time  and  budget  concerns. 

Value  of  Certain  Analyses.  The  value  of  some  analyses  was  questioned  by  the  reviewers.  The 
predictive  workload  analysis  and,  to  a  lesser  extent,  the  task/timeline  analysis,  were  specifically 
mentioned  as  not  providing  useful  information  in  the  past.  Most  reviewers  stated  that  their  crew 
system  groups  include  both  experienced  engineers  and  operational  crew  members  who  have 
knowledge  of  mission  choke  point  areas  and  the  time  required  to  perform  tasks  in  sequence.  They 
further  asserted  that  the  only  real  reason  for  performing  many  analyses  was  to  satisfy  a  Contract 
Data  Requirements  List  (CDRL)  deliverable. 

The  ability  to  perform  analysis  tasks  more  quickly  and  with  more  validity  was  received  with  a 
“show  me”  attitude.  Several  of  the  reviewers  stated  that  if  a  valid,  customer-endorsed  process  with 
practical  support  tools  were  provided,  confidence  in  the  value  of  these  analysis  activities  would  be 
improved  but  not  assured.  In  the  event  that  the  military  acquisition  offices  (e.g.,  SPOs)  request 
this  type  of  analysis,  the  CSDP  and  the  CDS  Toolset  should  expedite  the  work. 
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Iteration  of  Analyses  with  Design  and  Evaluation.  Those  reviewers  who  agreed  with  the  need  to 
perform  the  crew  system  analysis  tasks  suggested  that  the  CSDP  should  incorporate  a  greater 
amount  of  iteration  in  the  design  and  evaluation  phases  of  a  project.  That  is,  more  of  the 
information  developed  to  refine  the  cockpit  design  should  flow  through  the  iterative  cycles 
identified  in  the  CSDP.  This  could  be  accomplished  by  adding  an  iteration  reminder  to  the  already 
developed  cycles. 

Common  Database  and  Data  File  Information.  The  reviewers  were  pleased  with  the  fact  that  the 
CSDP  was  being  developed  so  that  data  could  be  accessed  for  each  analysis  task  from  a  common 
database.  The  reviewers  saw  that  this  information  flow  within  and  between  activities  of  the  CSDP 
would  save  time  when  performing  various  analyses.  The  reviewers  stated  that  if  they  were 
required  to  perform  the  full  scope  of  the  CSDP  analyses,  a  shared  database  of  common  information 
would  result  in  tremendous  time  savings  over  the  life  of  the  project. 

Crew  System  Design 

The  reviewers  stated  that  Crew  System  Design  is  the  most  critical  of  the  five  CSDP  categories 
because  of  the  accountability  that  is  necessary  in  the  design  of  each  cockpit  subsystem.  They  also 
stated  that  the  majority  of  the  time  and  budget  allocation  is  devoted  to  the  design  phase  of  each 
project. 

Iteration  of  Design  Cycles.  The  reviewers  agreed  with  the  design  iteration  aspects  of  the  CSDP. 
In  fact,  many  stated  that  more  formal  definitions  of  the  design  iterations  should  be  developed  as 
major  facets  of  the  detailed  CSDP  procedures.  Each  company  stated  that  the  cornerstone  of 
development  was  the  iteration  of  rapid  prototyping  activities,  which  is  performed  throughout 
design,  development,  and  early  evaluation.  The  reviewers  felt  that  a  greater  number  of  completed 
iterations  would  result  in  a  higher  quality  design. 

Activity  Development.  The  design  portion  of  the  CSDP  was  found  to  have  a  thorough  layout  of 
activities  relating  to  the  design  of  the  PVI.  However,  as  discussed  in  the  Crew  System  Analysis 
section,  the  reviewers  expressed  concern  that  the  CSDP  did  not  sufficiently  cover  other  factors 
such  as  life  support,  escape  and  reach,  cockpit  layout,  canopy  and  structural  analysis,  ray  tracing, 
and  cockpit  lighting. 

Rapid  Prototyping.  Rapid  prototyping  was  the  most  widely  cited  technique  used  to  assist  in  the 
design  and  evaluation  of  controls  and  displays.  Each  company  surveyed  has  a  device  (or  devices) 
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that  permits  near-real-time  representation  of  cockpit  controls  and  displays.  Typically,  rapid 
prototyping  simulators  may  lack  full  system  functionality  but  have  adequate  responses  provided  in 
interactive  simulation.  The  design  is  described  in  mechanization  documents,  which  usually  consist 
of  written  and  graphical  information  about  implementation  concepts.  The  design  engineers  write 
the  mechanization  document  and  then  work  with  software  engineers  to  implement  the  designs  on 
graphical  engines  such  as  Silicon  Graphics  or  Sun  workstations.  Once  working,  the  designs  can 
be  exercised  in  representative,  but  often  rudimentary,  mission  tasking  situations.  Evaluations  are 
then  conducted,  usually  by  members  of  the  design  team  who  have  an  operational  background.  The 
data  are  almost  entirely  subjective,  and  design  changes  are  quickly  made  in  the  rapid  prototyping 
simulator.  Rapid  prototyping  was  considered  a  valuable  and  extensively  used  element  of 
industry’s  current  process  for  cockpit  design. 

There  are  two  areas  where  the  CCCD  approach  might  improve  the  rapid  prototyping  technique 
used  by  industry.  The  first  area  involves  introducing  methods  to  objectively  evaluate  rapidly 
prototyped  design  solutions  in  order  to  augment  the  current  reliance  on  subjective,  qualitative 
assessments.  The  reviewers  were  concerned  that  objective  methods  might  impose  restrictions  that 
would  limit  the  quickness  of  the  process  (e.g.,  by  requiring  numerous  trials,  multiple  subjects,  or 
extensive  data  analysis).  If  the  objective  methods  did  not  unduly  compromise  the  speed  of  the 
process,  the  users  would  agree  to  try  them. 

The  second  area  includes  the  development  of  a  prototyping  system  that  does  not  overly  rely  on  a 
small  cadre  of  highly  skilled  programmers.  Currently,  the  industry  cockpit  simulators  typically 
rely  on  a  few  skilled  programmers  to  implement  the  cockpit  mechanizations;  therefore,  the 
reviewers  were  receptive  to  ideas  that  might  lessen  the  impact  of  losing  a  key  individual. 
However,  since  the  companies  already  possess  rapid-prototyping  systems,  they  appear  to  be 
reluctant  to  endorse  near-term  investments  in  a  new  system  (including  equipment  and  training 
costs)  for  far-term  advantages  of  reduced  reliance  on  key  individuals.  To  be  useful  to  industry, 
any  development  in  this  area  should  be  compatible  with  the  cockpit  simulators,  or  rapid 
prototyping  systems,  in  use. 

Design  Trade-off  Methodology.  The  formal  design  trade-off  methodology  within  the  CSDP  is 
based  on  Quality  Function  Deployment  (QFD).  The  basic  premise  is  similar  to  classical  system 
engineering.  Both  methods  require  engineers  to  perform  trade-offs  of  capabilities  versus 
implementation  costs  by  weighing  the  importance  of  the  trade-offs  and  deriving  the  best  solutions. 
The  QFD  methodology  employs  a  series  of  matrices  to  express  the  system  trade-offs.  The  CSDP, 
with  its  design  iteration  cycles,  depends  on  the  effective  use  of  trade-off  methods.  A  few  of  the 


12 


reviewers  were  experienced  (both  formally  and  informally)  in  using  the  QFD  methodology  and 
reported  that  the  results  were  useful.  They  stated  that,  while  it  takes  considerable  time  to  perform, 
ultimately  the  use  of  QFD  saves  time  by  providing  a  means  to  elicit  a  full  description  and  an 
interpretation  of  the  critical  facets  of  the  design.  In  this  way,  QFD  forces  the  design  team  to  justify 
the  design  features  in  terms  of  requirements.  This  justification  results  in  more  defensible,  higher 
quality  designs,  and  aids  design  traceability. 

Multidisciplined  Approach  with  CSDP.  The  reviewers  seem  receptive  to  a  more  formalized 
approach  to  guide  the  operation  of  inter-disciplinary  working  groups.  Since  each  company  has 
varied  organizational  structures,  it  was  suggested  that  the  CSDP  formally  recognize  the  use  of 
multidisciplined  working  groups.  Those  organizations  using  IPTs  would  benefit  from  the  explicit 
recognition  of  working  groups  within  the  CSDP  to  emphasize  the  DPT  function.  The  emphasis, 
especially  while  doing  design  work  on  the  CSDP,  will  be  on  the  trade-off  of  the  evolving 
requirements  against  the  implementation  impacts,  which  will  drive  the  cockpit  and  all  associated 
subsystems.  The  product  will  be  a  single  integrated  design  for  the  crew  station  (including 
avionics,  structures,  and  PVI)  that  is  expressed  in  the  mechanization  documents  and  specifications. 

Mechanizing  the  Pilot-Vehicle  Interface.  The  reviewers  were  engineers  with  diverse  backgrounds 
in  various  aspects  of  cockpit  design.  Some  of  them  were  responsible  for  designing  controls  and 
displays  and  writing  the  required  specifications  for  implementation.  Others  were  only  contributors 
to  the  design  and  implementation.  In  all  cases,  however,  the  engineers  seemed  to  require  some 
assistance  with  the  detailed  description  of  system  interfaces  within  the  cockpit.  The  discussions 
focused  on  the  ability  to  “speak  the  same  language”  as  the  subsystem  engineers  and  the  software 
engineers  when  discussing  either  simulation  or  actual  system  implementation.  The  CSDP  supports 
these  functions  through  the  use  of  mechanization  logic  diagrams  to  prepare  both  the  designs  and 
the  specifications  necessary  to  implement  the  design  interfaces.  Several  of  the  reviewers  have  used 
mechanization  logic  trees  and  reported  success  in  getting  designs  implemented  and  documented 
properly.  All  were  in  favor  of  the  inclusion  and  exploitation  of  mechanization  logic  trees  for  the 
development  of  controls  and  displays. 

Traceability  of  Design.  The  CSDP  requirement  for  design  traceability  left  the  reviewers  leery.  The 
discussions  centered  on  the  actual  need  for  tracing  the  rationale  and  history  of  cockpit  design 
features  (versus  the  current  practice  of  tracing  requirements,  rather  than  design  decisions,  as  part  of 
configuration  management).  The  reviewers  would  welcome  a  better  definition  of  how  much  detail 
the  government  wants  to  see  in  traceability  documentation. 
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Most  companies  will  be  providing  cockpit  design  traceability  information  for  the  SPOs.  Given  the 
choice,  the  companies  would  not  produce  more  than  current  documentation  requirements,  which 
consist  of  cockpit  specifications,  CDRL  submittals,  and  working  group  notes.  When  the 
government  team  clarified  that  the  CSDP  traceability  approach  would  enable  them  to  trace  specific 
requirements  through  design  implementation  and  test  verification,  the  reviewers  seemed  to  be 
encouraged  that  such  an  approach  would  be  useful  if  it  did  not  impose  the  significant  burden  of 
additional  documentation. 

The  specifics  of  requirements  traceability  seem  to  be  limited  to  the  “top”  and  “medium”  levels  of 
the  design.  This  allows  for  traceability  to  be  handled  both  at  a  major  functional  level  (e.g.,  why  a 
certain  number  of  displays  are  needed),  as  well  as  at  a  medium  level  (e.g.,  why  targeting 
information  on  all  displays  looks  the  way  it  does).  The  reviewers  did  not  want  to  go  to  the  lowest 
level  (e.g.,  why  certain  symbology  was  selected,  or  why  a  routine  switch  is  configured  a  certain 
way),  because  of  the  multitude  of  minor  changes  that  are  usually  made,  and  the  sheer  number  of 
items  within  a  cockpit.  They  indicated  that  little  would  be  gained  by  tracing  the  design  to  the 
lowest  level.  The  reviewers  requested  that  documentation  requirements  for  traceability  be  kept  to  a 
minimum,  and  not  require  tracing  to  the  component  level. 

There  was  apparent  skepticism  that  the  CSDP  emphasis  on  traceability  would  greatly  enhance  the 
ability  to  control  cockpit  designs.  However,  the  reviewers  noted  that  such  traceability  records 
would  assist  the  company  in  defending  its  current  design  by  establishing  a  direct,  logical 
connection  between  mission  requirements  and  system  features.  Additionally,  the  reviewers  agreed 
that  traceability  records  would  provide  a  design  history  database  to  filter  the  suggestions  submitted 
by  engineers  who  are  new  to  the  project.  The  records  could  be  used  to  eliminate  proposed 
enhancements  that  were  tried  before,  or  that  simply  do  not  meet  the  requirements  trail  as  it  has 
developed.  Finally,  the  reviewers  acknowledged  the  value  of  recording  (and  being  able  to  retrieve) 
the  traceability  information  for  use  in  cockpit  modification  programs  (e.g.,  block  changes). 

Crew  System  Evaluation 

Based  on  the  survey,  the  Crew  System  Evaluation  category  of  the  CSDP  requires  the  least  amount 
of  modification.  However,  when  the  reviewers  were  asked  specifically  about  improving  the 
evaluation  process  and  ensuring  the  validity  of  this  portion  of  the  CSDP,  they  provided  the  insights 
that  follow. 
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Simulation  and  Rapid  Prototyping  for  Cockpit  Evaluation.  The  overwhelming  finding  with  regard 
to  the  Crew  System  Evaluation  category  was  that  the  companies  use  several  types  of  interactive 
Pilot-in-the-Loop  (PIL)  simulation.  The  types  of  PIL  simulation  in  use  are:  rapid  prototyping, 
part-task,  full-task,  and  part-mission  testing.  Each  company  expressed  confidence  in  the  ability  to 
test  PVI  functions  within  each  category  of  PIL  testing.  Each  uses  a  combination  of  subjective  and 
objective  performance  measures,  but  the  great  majority  of  the  evaluations  are  done  subjectively. 
One  company  has  implemented  physiological  monitoring  systems.  The  actual  benefit  of  these 
evaluations  does  not  seem  to  be  as  clear  as  the  ability  to  perform  them.  That  is,  all  companies  felt 
confident  that  their  methods  of  planning  and  conducting  cockpit  simulator  studies  were  well 
thought-out,  but  the  results  from  the  studies  were  not  as  conclusive  as  desired.  In  summary,  the 
current  usage  relies  primarily  on  subjective  evaluation  of  the  cockpit  with  little  use  of  quantitative 
simulation  results  for  making  design  judgments. 

The  subject  of  rapid  prototyping  evaluation  was  enthusiastically  discussed  because  all  companies 
found  this  activity  to  be  the  most  prominent  and  useful  technique  for  design  evaluation  of  the  PVI. 
Rapid  prototyping  provides  a  relatively  quick  and  inexpensive  way  to  present  design  ideas  to  the 
operational  aircrews  and  design  engineers.  The  bulk  of  the  testing  is  conducted  informally  and 
typically  elicits  the  subjective  opinions  of  aircrew  personnel  from  several  sources,  such  as  the 
design  team,  the  overall  project  personnel,  and  the  customer.  The  reviewers  seem  to  be  satisfied 
with  the  results  of  these  evaluations  because  they  can  quickly  resolve  PVI  issues  prior  to  beginning 
expensive  software  development  for  higher  fidelity  simulation  (i.e.,  dome  simulators  or  avionics 
hot  benches).  Rapid  prototyping  also  furnishes  an  easy  and  affordable  way  to  develop  traceability 
of  the  PVI  design  and  to  help  document  user  acceptance. 

The  ability  to  plan  and  perform  all  types  of  PIL  simulator  evaluation  was  claimed  by  each  of  the 
companies.  The  majority  of  the  reviewers  had  previously  used  objective  and  subjective  measures 
to  evaluate  crew  performance,  workload,  and  situation  awareness  within  the  cockpit.  Some  had 
performed  PIL  simulations  with  physiological  measures  to  augment  the  other  data,  and  others  had 
used  derived  Measures  of  Effectiveness  (MOEs),  such  as  calculations  of  lethality  and  survivability, 
associated  with  the  type  of  aircraft  under  development.  In  summary,  most  of  the  evaluators 
concluded  that  the  best  evaluation  measures  were  formal  subjective  comments  obtained  directly 
from  the  aircrews,  even  though  objective  test  data  had  been  collected.  The  reviewers  agreed  that, 
in  current  industry  practice,  the  subjective  comments  of  the  aircrews  have  the  greatest  impact  on 
design  changes. 
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With  respect  to  a  simulation  test  planning  capability  within  the  CSDP,  the  majority  of  the  reviewers 
wanted  to  know  more  about  what  would  be  available  and  how  it  would  be  used.  A  SPO-endorsed 
planning  capability  for  PIL-based  evaluations  seemed  attractive.  It  was  explained  that  the  test 
planning  function  would  provide  the  capability  to  plan  any  type  of  PIL  evaluation  with  an 
appropriate  set  of  measures,  metrics,  and  reporting  techniques  advocated  by  the  government. 
Moreover,  a  database  of  additional  information  regarding  the  specifics  of  test  set-up,  training, 
operations,  and  data  analysis  techniques  could  be  accessed  when  required.  Finally,  it  was 
recognized  that  it  would  be  advantageous  to  use  the  same  test  planning  methods  for  both  simulation 
test  and  flight  test. 

Reliance  Upon  Software  Engineers.  Industry’s  current  ability  to  perform  most  PIL  evaluations 
seems  to  rely  on  a  small  cadre  of  software  engineers  who  know  how  to  rapidly  configure  the  PEL 
simulator  (see  Rapid  Prototyping-Crew  System  Design  section).  Occasionally,  the  software 
personnel  are  assisted  by  hardware  engineers,  but  the  reconfiguration  is  done  almost  exclusively  in 
the  software  group.  All  reviewers  admitted  that  the  loss  of  one  or  two  key  programmers  could 
essentially  bring  a  cockpit  project  to  a  halt.  When  asked  if  they  would  use  a  more  software- 
independent  method  of  designing  and  prototyping  displays  and  controls,  if  available,  the  reviewers 
replied  that  they  would  rather  continue  the  current  methods  than  invest  in  another  type  of  system. 

On  the  other  hand,  the  reviewers  agreed  that  the  matter  of  simulator  construction,  operation,  and 
maintenance  is  best  left  in  the  capable  hands  of  simulation  departments.  The  reviewers  noted  that 
current  simulator  operations  make  the  best  use  of  the  company  assets  and  do  not  inhibit  crew 
station  designers  from  planning  and  performing  the  needed  PIL  tests.  Typically,  simulator  assets 
are  shared  with  other  disciplines  such  as  avionics,  operations  analysis,  systems  engineering,  flight 
controls,  marketing,  public  relations,  and  others.  However,  the  design  team  usually  performs  the 
lead  role  in  the  simulator  cockpit  evaluations;  an  exception  is  the  detailed  design  of  the  flight 
control  system,  which  is  usually  handled  by  the  flight  controls  group. 

Other  Testing  Activities.  Several  of  the  reviewers  pointed  out  that  the  CSDP  did  not  adequately 
depict  the  nature  of  crew  system  evaluations  in  the  areas  of  cockpit  accommodation  (seating, 
egress,  reach,  vision),  life  support,  escape,  and  lighting.  Many  of  these  areas  are  typically 
subcontracted  and  the  associated  verification  testing  is  often  accomplished  in  the  subcontractor 
facility  or  in  a  system  integration  or  avionics  integration  laboratory  over  which  the  design  team  has 
little  control.  The  ramifications  of  these  evaluations  could  drastically  alter  cockpit  development  and 
should  be  included  in  the  design  trade-off  and  iteration  cycle,  and  should  be  recognized  during 
early  program  planning. 
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Evaluation  Timing.  Some  companies  suggested  that  the  CSDP  should  call  for  the  earlier  use  of 
evaluation  and  simulation  to  support  the  process  of  the  cockpit  (and  air  vehicle)  specification. 
(Note  that  “simulation”  in  this  context  refers  only  to  real-time,  PIL  simulation.)  The  reviewers 
asserted  that  early  in  the  process,  the  requirements  analysis  and  design  efforts  often  begin 
simultaneously,  so  that  simulation  activities  occur  much  sooner  than  presently  shown.  This  would 
appear  to  be  more  the  case  for  Concept  Exploration  than  for  later  phases.  The  reviewers  also 
remarked  that  simulation  as  a  means  of  investigating  excursions  from  the  specified  aircraft 
requirements  would  be  useful  for  the  CSDP. 

The  reviewers  agreed  that  an  evaluation  of  the  crew  station,  early  in  the  design  cycle,  would  help  to 
bring  problems  to  the  surface,  and  would  allow  time  for  correction.  Using  the  CSDP,  this 
evaluation  is  a  formal  part  of  the  crew-centered  methodology  under  which  the  cockpit  design 
follows  a  controlled  evaluation.  This  evaluation  starts  with  requirements  development,  continues 
with  analysis  and  design,  and  ends  by  verifying  that  the  requirements  were  met.  The  need  for 
early  cockpit  evaluation  must  be  tempered  by  the  need  for  an  orderly,  controlled,  and  traceable 
process  that  can  be  used  to  produce  a  dependable  cockpit  design. 

Prioritization  of  Cockpit  Design  Objectives 

The  reviewers  were  presented  with  a  list  of  22  candidate  objectives  of  the  CCCD  Program.  They 
were  asked  to  rank  each  objective  from  one  (most  important)  to  five  (least  important)  and  to  mark 
the  three  most  important  objectives  with  asterisks  (*)  and  the  three  least  important  with  checks  (V). 
The  objectives,  and  the  quantitative  results  obtained  from  the  reviewers,  are  listed  in  Table  2  in  the 
same  order  as  they  were  presented  to  the  reviewers.  The  number  of  ratings  per  objective  varies  in 
several  cases  because  some  reviewers  opted  not  to  prioritize  some  objectives. 

Several  of  the  objectives  are  quite  general,  with  no  direct  connection  to  specific  aspects  of  the 
CSDP  or  the  CDS  Toolset.  For  example,  the  reviewers  consistently  prioritized  Ensure  safety  in 
cockpit  design  as  an  important  objective.  However,  the  reviewers  stated  that  cockpit  safety  is  a 
“motherhood”  concern  that  is  always  given  top  billing,  but  has  no  direct  implication  for  the  process 
or  the  tools.  Therefore,  the  objectives  listed  in  Table  3  have  not  been  included  in  the  remainder  of 
this  report  because  they  do  not  require  specific  upgrades  of  the  CSDP  or  the  CDS  Toolset. 
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Table  2.  Objectives  and  Priorities  from  Survey  Ratings 


Ratings 

*1, *1,2,1, *1,1, 2, *1, *1,2 
*  1  ,*  1 ,2,1,*  l  ,3,*  1  ,*  1 . 1 
*1, *1, *1,1, 1,1, 2,1, 2, *1 
3, 2,4,4,*  1,*  1,4,2,*  1,1 

3.2, *1,3,3,2,2,*1,3,2 
3,V5,3,*1,3,3,*1,3,2,2 

3.3, *1,2,2,1,2,2,2,2 

3.2.3.2.3.3, *1,2,3,V3 

3.2.4. 2. 3. 2,  *1,2, 2, *1 

3.2,  *1,2,3, 1,3, 2, 1,*1 

2.4. 3,  *1,1, 3, 4, 3, 2,1 
2,V4,5,1,*1,2,V4,3,2,1 

3. 3.5. 2. 3.4. 3. 4.2. 2 
V3,V4,4,3,V5, 3,1, 2,3,2 

3. 2.2. 4. 1,  V4, 3,  l, 4,2 

2.4,  ^5,4, 3,2,3,  >5,3,2 
V3,3,3,V5,V5,V4,2,3,V5,a/3 

3. 3.2.3. 4.3.3. 4.3. 2 
V3,2,*  1,4, 3,3, 5,3,4, 2 

3.2.4, >/5,4,3,5,2W4,2 

3.3, V5,V5,V5,V4,V4,V4,V5,V3 

1.3.2.2.2, V4,V4,2,2 


Objectives 

Translate  mission  requirements  into  cockpit  design  constraints. 
Emphasize  crew  capabilities  in  cockpit  design. 

Ensure  safety  in  cockpit  design. 

Trace  the  decisions  that  led  to  a  cockpit  design. 

Control  the  configuration  of  various  cockpit  designs. 

Visualize  design  alternatives  interactively. 

Compare  cockpit  alternatives  to  a  baseline  cockpit. 

Modify  and  evaluate  cockpit  designs  in  a  matter  of  days. 

Minimize  the  cost  of  each  cockpit  design  iteration. 

Maximize  the  quality  of  each  cockpit  design  iteration. 

Apply  validated  and  accepted  procedures  to  cockpit  design. 

Use  of  standardized,  comparable  measures  of  cockpit  design 
characteristics. 

Provide  an  integrated  suite  of  design  tools  to  assist  with  cockpit  design. 
Minimize  purchase/maintenance  cost  of  hardware  and  software  tools. 
Provide  a  communications  medium  for  a  cockpit  design  team. 

Store  design  data  and  lessons-leamed  in  a  central  repository. 

Reduce  reliance  upon  external  resources  during  cockpit  design. 

Aid  the  preparation  of  design  documents,  specifications,  and  reports. 

Aid  the  scheduling  of  analysis,  design,  test  activities  for  cockpit  projects. 
Facilitate  preparation  of  labor  estimates  for  crew  system  design  projects. 
Depict  a  graphical  status  of  the  cockpit  design  project. 

Organize  operational  knowledge  in  an  easy-to-understand  and  use  format. 


Average 

1.3 

1.3 
1.2 

2.3 
2.2 
2.6 
1.9 

2.5 
2.2 
1.9 

2.4 

2.5 
3.0 
3.0 

2.6 

3.3 

3.6 
3.0 
3.0 

3.4 
4.1 

2.4 
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Table  3.  Objectives  Which  Do  Not  Require  Upgrades 

Ensure  safety  in  cockpit  design. 

Maximize  the  quality  of  each  cockpit  design  iteration. 

Compare  cockpit  alternatives  to  a  baseline  cockpit. 

Reduce  reliance  upon  external  resources  during  cockpit  design. 

Elimination  of  the  four  general  objectives  listed  in  Table  3  results  in  the  list  of  objectives  provided 
in  Table  4.  The  objectives  in  Table  4  are  listed  in  descending  order  of  importance  as  they  were 
ranked  by  the  reviewers.  The  additional  discriminators  (asterisks  and  check  marks)  were  used  as 
“tie  breakers”  to  order  the  closely  ranked  objectives. 

Table  4.  Objectives  in  Descending  Order  of  Importance 

Translate  mission  requirements  into  cockpit  design  constraints. 

Emphasize  crew  capabilities  in  cockpit  design. 

Control  the  configuration  of  various  cockpit  designs. 

Minimize  the  cost  of  each  cockpit  design  iteration. 

Trace  the  decisions  that  led  to  a  cockpit  design. 

Apply  validated  and  accepted  procedures  to  cockpit  design. 

Organize  operational  knowledge  in  an  easy-to-understand  and  use  format. 

Modify  and  evaluate  cockpit  designs  in  a  matter  of  days. 

Use  of  standardized,  comparable  measures  of  cockpit  design  characteristics. 

Visualize  design  alternatives  interactively. 

Provide  a  communications  medium  for  a  cockpit  design  team. 

Aid  the  scheduling  of  analysis,  design,  test  activities  for  cockpit  projects. 

Aid  the  preparation  of  design  documents,  specifications,  and  reports. 

Provide  an  integrated  suite  of  design  tools  to  assist  with  cockpit  design. 

Minimize  purchase/maintenance  cost  of  hardware  and  software  tools. 

Store  design  data  and  lessons-leamed  in  a  central  repository. 

Facilitate  preparation  of  labor  estimates  for  crew  system  design  projects. 

Depict  a  graphical  status  of  the  cockpit  design  project. 
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Summary  of  the  CSDP  Review 


The  results  of  the  prioritization  offer  meaningful  insights  into  probable  needs  of  industry,  subject 
to  two  caveats.  First,  the  rankings  were  done  prior  to  the  industry  site  visits.  Time  did  not  permit 
giving  the  reviewers  an  opportunity  to  revise  the  rankings  after  hearing  the  extensive  discussion  of 
the  CCCD  Program  objectives  and  accomplishments  to-date.  It  is  likely  that  the  additional 
information  could  have  influenced  the  ratings.  Second,  the  personnel  who  attended  the  on-site 
reviews  often  did  so  in  lieu  of,  or  in  addition  to,  the  personnel  who  reviewed  the  Industry  Review 
Package.  For  example,  at  one  company  the  reviewers  who  were  present  at  the  on-site  meeting 
emphasized  the  need  for  standard  measures  and  metrics.  However,  the  two  "package"  reviewers 
(who  could  not  attend  the  meeting)  had  placed  low  priority  on  this  objective.  Such  differences  of 
opinion  are  not  reflected  in  the  above  ratings  that  are  based  solely  on  written  responses  to  the 
Industry  Review  Package.  These  ratings  will  be  considered  in  future  CDS  upgrade  plans. 

CDS  TOOLSET 

Brief  summaries  describing  the  proposed  CDS  Toolset  were  given  to  the  reviewers  as  part  of  the 
Industry  Review  Package.  Details  about  the  functions  of  the  tools  were  intentionally  omitted  so  as 
not  to  bias  the  reviewers’  consideration  of  tool  utility.  As  expected,  many  of  the  reviewers 
requested  more  information  on  the  functions  and  capabilities  of  the  tools.  A  few  of  the  reviewers 
suggested  that  a  videotape  presentation  would  be  useful  to  highlight  the  tools  and  to  show  how 
they  support  CSDP  activities. 

The  reviewers  were  asked  to  prioritize  the  CDS  tools.  An  interesting  and  unanticipated  aspect  of 
this  prioritization  was  that  some  reviewers  ranked  the  tools  according  to  their  organization’s  need 
for  that  tool,  rather  than  for  the  importance  of  the  tool  in  supporting  the  development  process.  That 
is,  some  reviewers  ranked  a  tool  lower  if  it  was  similar  to  one  that  they  already  have,  and  ranked 
another  tool  higher  if  it  was  one  that  their  organization  needs.  This  approach  to  prioritization  was 
taken  into  consideration  when  analyzing  the  results. 

General  Comments  on  the  CDS  Toolset 

Several  comments  obtained  during  the  survey  apply  generally  to  all  of  the  CDS  Toolset.  These 
comments  are  summarized  below. 
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Tool  Validation.  An  extensive  discussion  centered  on  the  validation  of  the  CDS  Toolset.  One 
reviewer  indicated  that  the  CCCD  Program  would  be  providing  a  valuable  resource  to  the  design 
community  if  it  produced  an  established,  consistent  means  to  validate  candidate  tools.  He 
suggested  that  the  data  files  from  the  ongoing  CCCD  Field  Demonstrations  or  other  validation 
activities  be  preserved  and  distributed  to  serve  as  a  basis  for  future  evaluations.  It  was  recognized 
that,  while  this  may  be  a  good  idea  in  principle,  differences  in  tool  interfaces  may  make  it 
impractical. 

Piecewise  Development.  It  was  agreed  that,  while  an  integrated  toolset  might  be  ideal,  a  piece- 
wise  approach  to  the  development  of  the  tools  should  be  considered,  particularly  in  view  of  the 
current  defense  funding  environment.  This  opinion  was  borne  out  by  the  relatively  low  priority 
given  to  the  CDS  objective  Provide  an  integrated  suite  of  design  tools  to  assist  with  cockpit  design. 

User  Interface.  Given  the  widespread  support  for  a  piecewise  development,  reviewers 
acknowledged  that  the  user  interface  may  vary  from  tool  to  tool.  However,  the  desire  for  effective, 
user-friendly  interfaces  was  unanimous.  Intuitive,  easy-to-leam  interfaces  were  requested  to 
minimize  training  on  new  systems.  One  reviewer  stated  that  he  would  expect,  as  a  minimum,  a 
twofold  payback  on  the  time  spent  learning  a  new  tool. 

Several  companies  balanced  their  desire  for  user-friendly  interfaces  with  a  strong  desire  for 
functionality.  They  explained  that  if  a  tool  offers  a  useful  capability  but  a  somewhat  cumbersome 
interface,  it  will  be  used  nonetheless.  At  some  point  the  problems  with  usage  will  outweigh  the 
advantages  and  the  tool  will  no  longer  be  used.  It  seems  clear  that  the  CDS  Toolset  will  be  used, 
in  spite  of  a  suboptimal  interface,  if  it  provides  the  design  community  with  useful,  effective 
support. 

Global  Relational  Database.  Most  reviewers  suggested  that  all  tools  share  a  common  relational 
database.  This  suggestion  is  reflected  by  the  high  priorities  given  to  the  objectives  Trace  the 
decisions  that  led  to  a  cockpit  design  (priority  of  2.3)  and  Provide  a  communications  medium  for  a 
cockpit  design  team  (priority  of  2.6).  The  availability  of  a  common  database  would  also  help  to 
meet  the  objectives  of  Modify  and  evaluate  cockpit  designs  in  a  matter  of  days  (priority  of  2.5), 
and  Aid  in  the  preparation  of  design  documents,  specifications,  and  reports  (priority  of  3.0).  It  is 
also  closely  related  to  the  ability  to  Store  design  data  and  lessons-leamed  in  a  central  repository 
(priority  of  3.3). 
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Program  Planning  and  Scheduling  Tool  (PPST) 


The  CDS  objective  that  is  directly  related  to  the  PPST  is  Aid  the  scheduling  of  analysis,  design, 
test  activities  for  cockpit  projects.  This  objective  was  given  an  aggregate  priority  of  3  on  a  scale  of 
5  (with  5  being  low  priority)  by  the  personnel  surveyed.  There  are  two  reasons  for  this  relatively 
low  priority. 

First,  airframe  manufacturers  use  powerful  and  complex  program  planning  and  scheduling  tools, 
for  example,  Artemis,  a  commercial  software  product,  and  the  Integrated  Master  Information 
Control  System  (IMICS)  by  McDonnell  Douglas.  Usually,  because  of  the  complexity  of  these 
tools,  the  planning  and  scheduling  inputs  are  made  by  specially-trained  personnel.  That  is,  crew 
station  inputs  are  furnished  to  the  trained  operators  who  perform  data  entry  and  provide  the 
planning  analysis  reports.  Second,  on  small  projects,  PC-based  tools  (typically  MacProject  and 
MacSchedule)  have  been  found  to  be  satisfactory.  The  reviewers  preferred  commercial  off-the- 
shelf  products,  and  gave  PPST  a  relatively  low  priority. 

The  reviewers  remarked  that  automated  assistance  in  program  planning  is  unnecessary  because 
program  planning  personnel  already  know  how  to  prepare  schedules.  Several  reviewers  stated  that 
a  major  concern  with  automated  planning  systems  is  the  burden  of  maintaining  the  database  of 
reference  information  and  the  difficulty  and  costliness  of  data  entry.  One  feature  of  a  planning  and 
scheduling  tool  regarded  as  useful  would  be  the  capability  to  permit  users  to  exchange  PPST  files 
with  the  existing  corporate  planning  software.  However,  because  of  the  variety  of  corporate 
planning  systems  now  in  use,  the  development  of  such  a  tool  was  not  considered  a  high  priority 
for  the  CDS. 


Simulation  Test  Planning  Tool  (STPT) 

The  CDS  objective  Aid  the  scheduling  of  analysis,  design,  test  activities  for  cockpit  projects 
(priority  of  3)  applies  to  this  tool  as  well  as  the  PPST.  Two  other  objectives  relate  to  this  tool:  Use 
of  standardized,  comparable  measures  of  cockpit  design  characteristics  and  Store  design  data  and 
lessons-leamed  in  a  central  repository . 

The  reviewers  gave  the  test-planning  functionality  of  the  STPT  a  low  priority  because  it  is  their 
contention  that  they  already  understand  how  to  plan  simulation  tests,  including  the  selection  and 
definition  of  the  tests  and  the  required  resources.  The  STPT  was  characterized  as  a  “nice-to-have” 
tool,  but  one  that  is  nonessential.  However,  several  reviewers  requested  lessons-leamed  data 


about  specific  measures  that  would  be  a  feature  of  the  STPT,  and  would  relate  directly  to  the 
objective  Use  of  standardized,  comparable  measures  of  cockpit  design  characteristics  (priority  of 
2.5).  This  priority  of  2.5  was  given  in  the  written  review  and  appeared  to  be  ranked  higher  after 
face-to-face  discussions.  This  feature  would  also  relate  to  Store  design  data  and  lessons- learned  in 
a  central  repository  (priority  of  3.3). 

The  reviewers  also  expressed  an  interest  in  having  access  to  standard  procedures  for  evaluating 
cockpit  features  using  PIL  simulation.  These  procedures  might  be  similar  to  the  Structured  Test 
Procedures  for  flight  testing  contained  in  the  Test  Planning,  Analysis,  and  Evaluation  workstation 
that  was  developed  (separately  from  the  CDS)  for  the  CCCD  Program. 

Cockpit  Product  Tool  (CPT) 

The  reviewers  expressed  moderate  interest  in  the  CPT,  if  it  could:  (1)  provide  assistance  in 
defining  products  and  deliverables;  (2)  provide  assistance  in  generating  products  and  deliverables; 
and  (3)  inter-operate  with  the  corporate  document  preparation  systems.  A  priority  of  3.0  was 
given  to  the  objective  Aid  preparation  of  design  documents,  specifications,  and  reports. 

The  Computer-Aided  Acquisition  and  Logistics  System  (CALS)  initiative  did  not  appear  to  have  a 
significant  impact  on  the  requirements  for  a  CPT,  perhaps  because  CALS  has  had  little  or  no 
extension  to  cockpit  projects,  to  date.  Only  one  manufacturer  showed  an  awareness  of  CALS  and 
the  implications  of  CALS  compliance.  It  has  been  reported  that,  while  only  a  few  CALS  sites  are 
now  operational,  more  than  240  CALS  sites  will  be  operational  by  1995  to  help  industry  cut  costs 
and  manage  work  better  (Reference  2). 

The  current  software  tools  for  documentation  were  regarded  as  adequate  by  the  reviewers.  The 
PC-based  and  Macintosh-based  tools  such  as  Microsoft  Word,  MacDraw,  MacSchedule, 
PowerPoint,  and  Excel  are  often  used.  However,  the  ability  to  import  and  integrate  data  from  the 
analysis  and  evaluation  tools  was  considered  an  important  function  of  a  CPT. 

One  reviewer  stated  that  design  teams  usually  are  not  responsible  for  CDRL  reports  and  that  this 
tends  to  limit  the  perceived  importance  of  a  CPT.  He  stated  that  the  CPT  would  be  a  valued 
addition  if  it  fostered  the  definition  and  generation  of  cockpit  products  such  as  the  Cockpit 
Mechanization  Document. 
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Design  Traceability  Manager  (DTM) 


The  CDS  description  provided  in  the  Industry  Review  Package  did  not  describe  the  DTM  design 
and  functionality.  During  the  visits,  however,  the  essential  features  of  DTM  were  described, 
namely,  a  graphical  depiction  of  the  CSDP;  the  ability  to  launch  many  of  the  CDS  tools;  an 
electronic  logbook;  and  the  ability  to  trace  design  decisions.  Many  of  the  present  and  planned 
features  of  the  DTM,  such  as  the  ability  to  manage  project  contexts,  were  not  discussed.  Context 
management  would  relate  to  industry’s  highly  prioritized  requirement  to  Control  the  configuration 
of  various  cockpit  designs  (priority  of  2.2). 

There  was  a  strong  interest  in  the  DTM  features.  This  interest  stemmed  mainly  from  the  DTM 
traceability  feature,  as  reflected  in  the  high  priority  given  to  the  objective  Trace  the  decisions  that 
led  a  cockpit  design  (priority  of  2.3).  In  specifying  the  requirements  for  traceability,  reviewers 
expressed  concern  about  levying  additional  reporting  requirements  on  the  developer.  The 
reviewers  seemed  to  be  uncertain  about  how  the  traceability  data  would  be  used  to  determine  what 
aspects  of  the  design  to  track.  For  example,  should  the  rationale  behind  individual  symbols  be 
traced,  or  should  formats  be  displayed  in  their  entirety?  Simulation  reports  often  document  the 
rationale  behind  decisions,  but  it  was  noted  that  these  reports  are  not  readily  accessible  to  the 
design  team. 

All  reviewers  were  aware  of  the  government  interest  in  design  traceability  and  it  was  understood 
that  the  Air  Force  SPOs  advise  using  traceability  management  in  the  CDS  development.  The 
reviewers  emphasized  a  need  for  a  general-purpose  relational  database  that  can  retain  cockpit 
design  data  and  make  it  readily  available  to  the  design  team. 

At  one  company,  a  commercial  software  product  from  Ascent  Logic,  Incorporated  (RDD-100)  was 
being  examined  and  the  reviewers  mentioned  that  it  may  be  similar  to  the  DTM  software. 
Following  a  demonstration,  it  appeared  that  the  RDD-100  software  was  more  appropriate  for  use  in 
requirements  traceability  than  design  traceability.  This  software  also  had  features  that  would  not  be 
required  in  the  DTM. 


Concept  Mapping  Tool  (CMT) 

The  technique  of  concept  mapping  and  the  tool  to  support  it  (Reference  3)  received  interest  during 
the  review.  This  interest  was  manifested  in  the  high  priority  given  to  Organize  operational 
knowledge  in  an  easy-to-understand  and  easy-to-use  format  (priority  of  2.4).  The  reviewers 
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expressed  a  need  for  a  method  to  achieve  concurrence  among  Subject  Matter  Experts  (SMEs)  for 
operational  requirements,  which  would  be  supported  by  the  concept-mapping  technique  and  tool. 
In  addition,  the  reviewers  responded  positively  when  asked  if  it  would  be  advantageous  to  feed  a 
timeline-generation  process  with  the  results  of  the  concept  mapping  of  cockpit  tasks. 

The  majority  of  the  reviewers  were  not  familiar  with  concept  mapping  as  a  cockpit  design 
technique.  Those  personnel  familiar  with  concept  mapping  were  skeptical  about  its  value.  They 
found  the  results  from  its  use  to  be  dependent  on  the  person  performing  the  interview  and  analysis. 

Overall,  the  conclusion  was  that  concept  mapping  as  both  a  tool  and  a  technique  would  be  useful, 
particularly  early  in  a  project  as  a  means  of  clarifying  operational  requirements,  but  those  results 
must  be  used  wisely.  It  was  also  agreed  that  concept  mapping  would  have  particular  value  as  an 
“introductory  tool”  early  in  the  project,  to  acquaint  human  factors  and  system  design  personnel 
with  operational  procedures  and  issues.  The  reviewers  expressed  an  interest  in  the  developmental 
status  and  availability  of  the  Tool  for  Automated  Knowledge  Engineering  (TAKE,  Reference  3), 
which  is  currently  used  in  the  CDS,  and  remains  under  development  in  the  Armstrong 
Laboratory’s  Crew  System  Integration  Branch  (AL/CFHI). 

Mission  Decomposition  Tool  (MDTOOL) 

Several  of  the  companies  surveyed  expressed  a  strong  interest  in  MDTOOL  as  a  means  of 
describing  the  mission  and  decomposing  it  to  the  event  level.  This  interest  is  indicated  by  the  high 
priority  given  to  Translate  mission  requirements  into  cockpit  design  constraints  (priority  of  1.2). 

One  company  expressed  interest  in  a  mission  decomposition  tool  that  could  handle  Battlefield  Air 
Interdiction,  Offensive  Counter  Air,  and  several  other  types  of  missions.  The  reviewers  felt  that 
MDTOOL  would  improve  the  purely  manual  techniques  that  are  now  used.  The  reviewers  felt  that 
mission  engagement  models,  such  as  TAC  BRAWLER,  were  too  complicated  and  too  powerful 
for  the  mission  decomposition  that  is  needed  for  the  CSDP. 

Little  interest  was  shown  in  the  simulation-set-up  feature  of  MDTOOL.  This  was  probably  due  to 
the  idiosyncrasies  of  each  simulator  and  the  availability  of  existing  setup  tools  for  many  simulators. 
The  reviewers  noted  the  benefit  of  sensor  events  such  as  detection,  tracking,  and  launch  within 
MDTOOL  as  a  function  of  conditions  such  as  weather,  terrain  obscuration,  and  electronic  counter¬ 
measures  (ECM).  Assistance  with  analyzing  “what-ifs”  was  also  desired,  but  it  was  not  clear  if 
this  requirement  is  entirely  a  cockpit  development  issue. 
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Design  Trade -off  Tool  (DTT) 


During  the  industry  visits,  the  DTT  was  described  as  a  tool  that  can  be  used  to  perform  QFD 
(Quality  Functional  Deployment).  The  reviewers  had  relatively  little  knowledge  of  QFD  and  there 
was  substantial  interest  in  learning  more  about  this  approach,  the  QFD  Designer  software,  and  how 
these  tools  could  support  the  crew  system  development  process.  This  interest  is  reflected  by  the 
priority  of  1.9  given  to  the  Maximize  the  quality  of  each  cockpit  design  iteration  objective.  Most  of 
the  companies  already  use  a  matrix-type  approach  to  evaluate  design  trades,  but  do  not  use  the 
formal  QFD  method.  The  companies  use  commercial  spreadsheet  programs,  such  as  Excel,  to 
support  their  current  process. 

One  reviewer  stated  that  his  company  uses  part  of  the  QFD  approach  to  evaluate  design  trade-offs. 
He  reported  that  weighted  matrices  are  used,  but  that  the  effort  does  not  currently  progress  through 
the  “House  of  Quality”  formality  within  QFD.  The  QFD  approach  was  viewed  as  a  means  to 
accomplish  the  difficult  tasks  of  establishing  criteria,  weights,  and  interactions  early  in  the 
development  process.  Those  matrices  would  then  foster  faster  and  earlier  decision  making.  The 
reviewers  agreed  that  a  valuable  means  of  substantiating  design  decisions  would  be  to  be  able  to 
enter  the  results  from  the  QFD  tool  into  the  design  traceability  function. 

Mechanization  Logic  Tree  Tool  (MLTT) 

There  was  no  CDS  objective  that  related  directly  to  the  prioritization  of  a  tool  to  assist  in  the 
preparation  of  the  cockpit  mechanization  logic.  The  objective  Aid  preparation  of  design 
documents,  specifications,  and  reports  (priority  of  3.0)  is  perhaps  the  most  germane. 

The  site  visits  revealed  the  need  for  two  categories  of  logic-rendering  tools.  The  first  category 
comprises  simple  tools  to  express  a  functional  flow  in  terms  that  the  engineer  or  pilot  can  use  to 
represent  control  and  display  logic.  If/then/else,  case,  and  state  diagrams  would  be  included.  The 
second  category  comprises  tools  to  specify  the  avionic  software  and  hardware  design,  including 
data  dictionaries,  consistency  checks,  and  data  flow  diagrams. 

The  reviewers  agreed  that  only  the  first  category  of  capabilities  would  be  used  by  the  design  team. 
The  second  category  would  be  used  by  avionics  specialists  and  programmers,  not  crew  station 
designers.  At  one  company,  the  reviewers  cited  the  I-Logix  Statemate  software  as  being  in  the 
second  category  and  agreed  that  it  would  be  too  complex  for  use  by  the  design  team. 
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The  majority  of  reviewers  indicated  that  they  currently  do  not  have  tools  for  mechanization  logic 
preparation  and  therefore  gave  high  priority  to  the  development  of  a  mechanization  logic  tool. 
Currently,  the  mechanization  documents  (when  formally  prepared)  are  text  descriptions  of 
information  derived  from  design  specifications  and  supplemented  by  information  derived  from 
listings  of  the  simulator  software  code. 

The  reviewers  noted  that  some  of  the  newer  versions  of  control  and  display  prototyping  tools,  such 
as  the  latest  version  of  the  Virtual  Avionics  Prototyping  System  (VAPS),  can  generate  Ada  or  C 
code.  This  code  can  then  be  processed  by  a  reverse  engineering  tool  to  provide  flowcharts  or  other 
expressions  of  code  logic.  This  would  appear  to  be  a  cumbersome,  but  possibly  useful  means  of 
obtaining  logic  specifications.  While  there  seems  to  be  possible  value  in  the  automatic  generation 
of  code,  its  clear  use  for  crew  station  design  appears  unproven. 

Several  reviewers  emphasized  the  need  to  provide  a  specification  to  programmers  prior  to 
beginning  code  development.  Currently,  programmers  are  provided  only  a  two  or  three  page  text 
description,  but  may  not  be  provided  flow  charts.  Agreement  was  reached  that  it  would  be  useful 
to  have  a  tool  such  as  MLTT  to  provide  programmers  with  additional  material  in  the  form  of  flow 
charts  and  to  serve  as  a  guide  to  the  controls  and  displays  prototyping  process. 

The  reviewers  agreed  that  it  would  be  advantageous  to  electronically  export  the  early  logic 
specifications  from  the  design  team  to  the  system  used  by  the  avionics  engineers.  Several 
reviewers  identified  the  need  for  MLTT  to  assist  in  integrating  the  cockpit  control  and  display  logic 
into  the  aircraft,  including  the  air  vehicle  bus  data.  Often  the  cockpit  designers  assume  the 
availability  of  data  and  controls  that  the  avionics  engineers  later  find  difficult  or  impossible  to 
provide. 


Timeline  Management  Tool  (TMT) 

During  the  survey,  the  TMT  was  presented  as  an  aggregation  of  several  of  the  tools  that  were 
separately  identified  in  the  Industry  Review  Package.  These  tools  included:  the  Information  and 
Control  Requirements  Analysis  Tool  (ICRAT),  the  Function  Flow  Analysis  Tool  (FFAT),  and  the 
Function  Allocation  Trade  Analysis  Tool  (FAT AT).  These  tools  permit  the  analyst  to  populate  a 
mission  timeline  with  functions  and  tasks  (or  whatever  taxonomy  is  most  appropriate)  and  to 
extract  information  and  control  requirements.  Additionally,  the  populated  timeline  can  be  used  as 
input  to  taskload  and  workload  prediction  tools. 
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Much  of  the  discussion  during  the  industry  visits  focused  on  the  utility  and  validity  of  the  timeline 
analysis  functions.  The  reviewers  gave  high  ratings  to  the  CDS  objectives  that  relate  to  TMT. 
These  objectives  include:  Translate  mission  requirements  into  cockpit  design  constraints  (priority 
of  1.3);  Emphasize  crew  capabilities  in  cockpit  design  (priority  of  1.3);  Compare  cockpit 
alternatives  to  a  baseline  cockpit  (priority  of  1.9);  and  Minimize  the  cost  of  each  cockpit  design 
iteration  (priority  of  2.2).  While  there  was  a  perceived  need  for  timeline  information,  the  reviewers 
expressed  reservations  about  the  utility  and  validity  of  timeline  analysis.  The  interest  in  timeline 
analysis  tools  was  divided  generally  according  to  the  reviewers’  backgrounds.  Human  factors 
personnel  tended  to  support  the  use  of  timeline  analysis,  while  operational  analysts  tended  to  favor 
subjective  evaluations  over  analytical  techniques. 

Several  human  factors  analysts  stated  that  if  the  task  timeline  analysis  tool  was  proven  to  be 
reliable,  they  would  use  it.  As  one  individual  said,  anything  that  helps  with  “crunching”  the  data 
and  ranking  alternatives  (to  screen  the  most  promising  concepts  for  subsequent  simulation)  would 
be  useful.  The  industry  approach  to  evaluation  was  generally  described  by  most  reviewers  as 
essentially  subjective  and  qualitative  in  nature.  While  they  were  sometimes  criticized  for  this 
approach  by  managers,  several  operational  analysts  were  adamant  that  nothing  practical  could  be 
developed  based  upon  quantitative,  objective  data.  They  tended  to  have  little  use  for  predictive 
analysis,  favoring  analysis  of  experiential  data.  However,  several  operational  analysts  supported 
the  use  of  timeline  analysis  as  a  means  of  function  allocation  and  information  requirements 
analysis. 

In  summary,  it  appears  that  there  is  a  need  for  valid  and  reliable  timeline  analysis  tools,  and  that  in 
the  past,  such  tools  were  less  than  satisfactory.  Human  factors  engineers  appear  to  be  willing  and 
eager  to  apply  timeline  analysis  tools  if  they  are  endorsed  by  the  customer  and  if  timeline  analysis 
activities  are  supported  by  management. 

Workload  Analysis  Tool  (WAT) 

Many  reviewers  see  the  measurement  of  pilot  and  crew  workload  as  a  major  challenge  because 
there  is  no  accepted  set  of  measures  or  techniques.  Among  the  reviewers,  the  analytic 
predictability  of  workload  was  widely  questioned  by  the  operational  analysts.  The  analysts  stated 
that,  while  subjective  assessment  is  often  criticized,  it  remains  the  only  suitable  way  to  estimate 
workload  at  the  present  time  because  of  the  absence  of  other  valid,  accepted  techniques. 
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On  most  major  aircraft  programs,  the  crew  system  analysts  do  not  attempt  to  predict  workload 
using  quantitative  analytical  techniques.  Instead,  pilots  are  given  representations  of  the  mission 
(event  timelines,  task  timelines,  and  cockpit  concepts  in  static  or  dynamic  forms)  and  are  asked  to 
assess  the  projected  workload.  When  PIL  simulators  are  available,  performance  is  usually  judged 
using  time  and  accuracy  measures.  Workload  is  then  assumed  to  be  acceptable  as  long  as 
performance  is  satisfactory,  and  the  subjective  assessment  of  performance  is  also  acceptable.  The 
Subjective  Workload  Assessment  Technique  (SWAT)  is  sometimes  used  and  is  an  effective  means 
of  treating  subjective  data  in  a  quantitative  fashion,  but  simulator  pilots  sometimes  complain  about 
its  intrusiveness. 

As  with  the  timeline  analysis  tools,  the  majority  of  human  factors  analysts  surveyed  recognize  the 
need  for  valid  and  acceptable  measures  of  predicted  workload.  One  individual  referred  to 
workload  analysis  tools  as  a  “necessary  evil.”  Workload  is  frequently  cited  as  a  major  cockpit 
design  driver  and,  without  a  reliable  measure  of  this  design  parameter,  the  design  team  is  unable  to 
identify  an  effective  cockpit  design  early  in  the  development  process.  The  areas  of  workload 
prediction  and  measurement  are  important,  and  tools  that  support  workload  assessment  were 
considered  necessary  for  the  CDS. 

The  reviewers  suggested  that  the  integration  of  the  workload  analysis  tool  with  the  timeline 
analysis  tool  would  provide  an  efficient  and  useful  basis  for  analyzing  workload.  None  of  the 
companies  indicated  that  they  have  found  an  existing  workload  analysis  tool  that  is  acceptable.  In 
several  cases,  a  discussion  of  Sequitur’s  Workload  Analysis  System  (SWAS)  was  initiated.  The 
SWAS  is  a  commercial  software  tool  that  estimates  the  amount  of  workload  based  on  the  time 
available  for  individual  aircrew  tasks  versus  the  time  required  to  perform  them.  The  reviewers  had 
little  knowledge  of  SWAS  (one  company  had  used  the  software  in  an  aircraft  development 
program),  and  they  expressed  a  guarded  interest  in  its  use.  The  design  community  is  clearly 
awaiting  reasonable  proof  that  SWAS,  or  any  workload  analysis  tool,  is  a  valid  and  acceptable 
solution  to  workload  and  timeline  analyses  requirements. 

Crew  Station  Geometry  Tool  (CSGT) 

The  companies  have  established  computer-aided  design  (CAD)  capabilities  for  crew  station 
geometry  design  and  specification.  The  tools  include  CATIA  (trade  name  for  the  Dassault 
CAD/CAM  package).  Vision,  Computer-Aided  Design  and  Manufacturing  (CADAM),  Mechanical 
Computer-Aided  Design  (MCAD),  and  others.  While  these  tools  require  trained  operators,  they 
seem  to  provide  the  necessary  support  for  the  cockpit  development  process.  Other  cockpit 
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geometry-related  analyses,  such  as  ray  tracing  (for  vision  assessment)  and  canopy  reflection  are 
also  currently  performed  using  complex  tools  that  require  trained  operators.  Because  most  of  the 
companies  use  their  own  established  systems,  they  see  no  need  for  the  CCCD  Program  to  develop 
a  new  tool. 


Computerized  Biomechanical  Man  Model  (COMBIMAN) 

The  review  indicated  a  marginal  interest  in  COMBIMAN  as  a  CDS  tool.  Several  reviewers  stated 
that  they  seldom  design  totally  new  cockpits  and  would  have  limited  use  for  an  anthropometric 
tool.  The  need  in  this  area  is  satisfied  by  entering  the  anthropometric  dimensions  into  a  Computer- 
Aided  Design/Computer-Aided  Manufacturing  (CAD/CAM)  system  to  confirm  reachability  and 
clearance.  Static  mockups  (discussed  in  the  following  section)  are  used  to  augment  the  CAD/CAM 
analyses. 

Some  reviewers  had  used  earlier  versions  of  COMBIMAN  and  reported  problems  possibly  related 
to  its  having  been  compiled  for  an  older  mainframe  computer.  Interest  was  expressed  in  learning 
more  about  the  latest  version  of  COMBIMAN,  which  operates  under  CAD  software  using  a  UNIX 
workstation.  The  reviewers  identified  the  need  for  bivariate  or  multivariate  analysis  capabilities, 
and  were  interested  in  any  tool  that  would  provide  that  support.  The  emerging  standard  from  the 
Society  of  Automotive  Engineering  (SAE),  to  ensure  compatibility  and  transferability  across 
biomechanical  models,  was  also  discussed. 

An  anthropometric  model  is  a  necessary  component  of  a  complete  cockpit  development  toolset,  and 
COMBIMAN  is  the  anthropometric  model  of  choice  for  the  CDS  Toolset.  In  light  of  the  above 
information,  it  appears  that  no  additional  development  of  COMBIMAN  is  needed  for  its  use  in  the 
CDS,  but  some  integration  of  the  model  within  the  CDS  Toolset  may  be  necessary. 

Control  and  Display  Development  Tool  (CDDT) 

The  companies  surveyed  already  have,  or  are  currently  developing,  control  and  display 
development  (rapid  prototyping)  tools.  While  there  are  commercial  off-the-shelf  systems  available, 
none  has  been  found  to  be  fully  acceptable  by  any  of  the  companies.  Instead,  the  companies  have 
chosen  to  implement  custom  solutions  using  the  C  or  C++  languages,  typically  running  on  Silicon 
Graphics  processors.  These  custom  systems  have  proven  to  be  useful  tools,  capable  of  providing 
rapid  response  to  short-term  needs. 
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In  each  company,  the  major  drawback  of  the  existing  CDDTs  was  the  need  for  skilled 
programmers  who  are  knowledgeable  in  the  software  and  its  libraries  of  cockpit  elements.  One 
company  described  these  engineers  as  possessing  unique  capabilities  not  only  in  programming,  but 
also  in  cockpit  design  (because  of  the  amount  of  work  done  in  this  area  over  time). 

Like  the  anthropometric  model,  the  CDDT  is  a  necessary  component  of  the  CDS  Toolset.  The 
CDDT  will  be  used  to  perform  cockpit  studies  and  to  validate  the  other  CDS  tools.  It  is  clear,  from 
the  industry  comments,  that  widespread  interest  in  a  new  CDDT  will  result  if  the  reliance  on  skilled 
programmers  can  be  substantially  reduced.  It  is  expected  that  the  eventual  industry  users  of  the 
CDS  would  prefer  to  adapt  the  existing  tools  (rather  than  replacing  them  with  new  software),  in 
part  to  retain  commonality  with  the  data  from  previous  tool  usage. 

Part-Task/Part-Mission  Simulators 

The  value  of  the  part-task/part-mission  simulator  as  an  evaluation  tool  was  emphasized  by  the 
reviewers.  All  of  the  participating  companies  already  possess  combinations  of  full-task  or  part- 
mission/part-task  simulators,  some  of  which  are  tied  to  rapid-prototyping  systems.  The  use  of 
these  simulators  is  directly  dependent  on  a  number  of  skilled  software  engineers.  There  does  not 
appear  to  be  a  high  level  of  interest  in  acquiring  new  or  different  simulators  unless  this  dependency 
on  specialized  software  engineers  can  be  significantly  reduced. 

The  reviewers  generally  indicated  that  they  would  find  little  use  for  hardware-reconfigurable 
(“erector  set”)  simulators.  For  totally  new  cockpits,  the  companies  develop  low-cost,  static 
mockups  featuring  plywood  and  cardboard  frames  with  foam  core-based  overlays  of  control  and 
display  panels.  These  mockups  can  be  generated  by  the  CAD/CAM  systems  and  attached  with 
Velcro  or  glue.  The  CAD/CAM  renderings  of  the  geometry  are  used  to  create  the  mockup  panels 
and  to  analyze  crew  ingress  and  emergency  egress.  Designers  also  use  the  mockups  to  elicit 
opinions  from  pilots  on  the  control  and  display  geometry.  In  later  phases  of  development,  the 
mockup  is  often  used  as  a  framework  to  house  controls  and  displays.  The  mechanization  of  these 
displays  is  usually  done  using  the  rapid  prototyping  systems  previously  described.  In  the  latter 
phases  of  development,  when  an  existing  cockpit  is  available,  there  is  no  need  for  hardware 
reconfigurability  because  the  geometry  has  been  firmly  established. 

Some  reviewers  indicated  that  hardware-reconfigurable  simulators  are  useful  in  a  laboratory  setting 
in  which  many  different  geometric  arrangements  must  be  evaluated  in  a  succession  of  studies.  One 
reviewer  stated  that  if  hardware-reconfigurable  simulators  enabled  infinite  placement  capabilities. 
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they  might  be  more  useful,  but  that  positioning  of  controls  and  displays  is  normally  too  constrained 
to  make  this  capability  important. 

In  summary,  extensive  development  of  a  new  cockpit  simulator,  as  part  of  an  industry  version  of 
the  CDS,  does  not  appear  to  be  warranted  because  of  the  limited  interest  in  such  a  simulator  by 
industry,  and  because  previously  established  cockpit  simulators  fulfill  the  needs  of  cockpit  design 
teams.  Currently,  the  CDS  uses  a  hardware-reconfigurable  cockpit  simulator  (the  Engineering 
Design  Simulator,  or  EDSIM)  to  verily  the  utility  of  both  the  CSDP  and  the  CDS  Toolset. 
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PROCESSING  ENVIRONMENT 


The  companies  surveyed  indicated  a  strong  desire  to  have  a  personal  computer  (PC),  Macintosh 
computer,  or  engineering  workstation  for  each  design  team  member.  The  prevalent  processing 
environment  in  industry  today  is  characterized  by  a  heterogeneous  mixture  of  IBM-compatible 
PCs,  Macintosh  computers,  Silicon  Graphics  Incorporated  (SGI)  workstations,  and  Sun 
workstations. 

The  need  to  share  data  files  was  emphasized.  The  preferred  solution  is  a  client/server  architecture 
in  which  PCs,  Macintoshes  and  workstations  are  networked  to  shared  resources.  The  central 
feature  of  the  network  is  one  or  more  file  servers,  which  permit  the  sharing  of  project  files  among 
the  design  team  members.  The  network  typically  would  provide  shared  access  to  printers, 
modems,  and  other  peripheral  devices. 

The  cv  processors  seem  to  be  preferred  as  simulator  hosts  for  cockpit  control  and  display 
prototyping.  For  example,  an  Onyx  processor  is  the  host  for  one  company’s  rapid-prototyping 
system,  and  an  SGI  4D/220VGX  is  the  central  processor  for  its  cockpit  development  simulator. 
There  appears  to  be  a  migration  toward  Unix-based  platforms,  in  general,  and  SGI  processors  in 
particular. 

Several  companies  emphasized  the  need  to  use  the  CDS  Toolset  in  a  classified  environment.  For 
classified  programs,  this  implies  that  the  hosting  platforms  would  usually  be  limited  to  those 
already  contained  in  the  classified  facility.  For  this  reason,  IBM-compatible  PCs,  or  Macintosh 
computers,  were  suggested  as  the  best  host  platforms,  because  they  tend  to  be  the  most  widely 
available.  Classified  processing  would  prevent  the  export  of  data  out  of  the  classified 
environment.  For  this  reason,  data  files  created  for  classified  programs  would  not  be  available  for 
re-use  outside  of  the  program  itself,  even  if  they  contain  only  unclassified  information. 
Additionally,  the  use  of  removable  media  such  as  floppy  disks  and  compact  disks  (CDs)  would  be 
the  preferred  choice  for  data  storage. 


CONCLUSIONS  AND  RECOMMENDATIONS 


The  review  was  worthwhile  for  three  reasons.  First  and  foremost,  it  provided  a  forum  for 
gathering  and  updating  the  CCCD-related  requirements  of  industry.  These  requirements  will 
provide  a  sound  basis  for  the  subsequent  prioritization  of  future  CCCD  activities.  Second,  it 
provided  an  excellent  means  to  present  the  current  status  of  the  CCCD  Program  to  interested 
members  of  the  cockpit  design  community.  Third,  it  offered  an  opportunity  to  gamer  the  interest 
and  the  support  of  the  cockpit  design  community.  After  each  session,  the  industry  reviewers  stated 
that  they  appreciated  the  chance  to  review  and  comment  on  the  evolving  products  of  the  CCCD 
Program,  and  affirmed  that  they  would  welcome  continued  involvement. 

The  industry  reviewers  remarked  that  the  new  approach  and  direction  being  taken  by  the  CCCD 
Program  was  “headed  down  the  right  path.”  Each  of  the  companies  had  constructive  suggestions 
for  enhancements  to  both  the  CSDP  and  the  CDS  Toolset.  The  challenge  for  the  program  in  the 
near  future  is  to  prioritize  candidate  CDS  enhancements,  and  to  focus  on  those  that  are  most 
valuable  to  both  industry  and  government.  This  approach  is  leading  to  the  development  of  a  crew- 
centered  cockpit  design  process  and  support  tools  that  will  assist  in  formulating  the  next-generation 
cockpits  or  cockpit  modifications.  The  industry  review  provided  a  valuable  insight  into  the  prioriti¬ 
zation  and  implementation  of  near-term  CSDP  and  CDS  development. 

The  establishment  of  Beta  Sites  for  the  installation  and  use  of  the  CSDP  and  the  CDS 
Toolset  is  one  of  the  primary  goals  of  the  current  phase  of  the  CCCD  program.  Based  in  part  on 
this  review,  the  CDS  Toolset  is  being  developed  with  a  piecewise  architecture  so  that  individual 
tools,  or  groups  of  tools,  can  be  used  as  they  become  mature.  The  CCCD  Program  Office 
welcomes  inquiries  from  potential  users  of  the  process  and  tools  described  in  this  report. 
Interested  industry  or  government  officials  can  contact  the  CCCD  Program  Office  concerning  the 
possibility  of  becoming  a  CCCD  Beta  Site. 
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APPENDIX  A 


DESCRIPTION  OF  THE  CREW-CENTERED  SYSTEM  DESIGN 

PROCESS 


SECTION  1.0:  INTRODUCTION  AND  BACKGROUND 


1 . 1  Introduction 

Welcome  to  the  world  of  Crew-Centered  Cockpit  Design.  This  guide  will  familiarize 
you  with  the  process,  procedures,  and  tools  that  are  in  advanced  development  under  the  Armstrong 
Laboratory’s  Crew-Centered  Cockpit  Design  (CCCD)  Program.  The  CCCD  Program  seeks  to 
establish  a  well-organized,  thorough  and  inspectable  process  for  cockpit  development,  along  with 
an  effective  set  of  the  needed  software  and  simulation  support  tools.  The  physical  product  of  the 
development  to  date  resides  in  a  self-contained,  integrated  Cockpit  Design  System,  or  CDS. 
The  CDS,  itself,  contains  a  retrievable  guide  to  CCCD’s  Crew-Centered  System  Design 
Process,  or  CSDP.  (The  terms  “CCCD  Process”  and  “CSDP”  are  used  interchangeably  in  this 
document.)  Recently,  we  enhanced  the  previously  delivered  CSDP  and  CDS,  and  now  seek  your 
suggestions  for  further  improvement.  This  document  is  intended  for  review  by  cockpit  specialists 
within  the  aircraft  industry  and  also  by  System  Program  Offices  (SPOs)  and  Government 
Laboratories. 

This  guide  to  the  CSDP  and  CDS  was  prepared  to  “walk”  you  through  our  support  system 
for  cockpit  development.  The  CDS  represents  R&D  work  that  has  been  underway  for  several 
years,  and  is  still  in  various  stages  of  development.  With  your  help,  we  will  continue  to 
incorporate  system-wide  enhancements  and  corrections  that  are  based  upon  user  needs. 

The  CCCD  Process  Overview  (Figure  A-l)  depicts  crew  system  design  as  an  interactive 
(connecting  lines)  and  iterative  (cyclical  arrows)  process.  We  define  the  crew-centered  cockpit 
design  process  as  a  logical  collection  of  activities  and  procedures  which  examine  the  role,  the 
functionality,  and  the  inter-operability  of  the  crew  in  conjunction  with  aircraft,  avionics,  flight 
control,  and  other  air  vehicle  systems  to  achieve  mission  goals.  Therefore,  mission  requirements, 
weapon  system  capabilities,  as  well  as  crew  capabilities  each  can  dramatically  affect  the  crew 
system  design  object  (i.e.,  the  cockpit).  Inherent  in  the  CCCD  Process  are  many  activities  that 
have  common  roots  in  past  cockpit  development  work.  We  understand  the  need  to  balance 
comprehensive  requirements  with  timely  performance.  Therefore,  this  process  description 
includes  a  discussion  of  design  tools  which  are  intended  to  dramatically  reduce  the  time  to  complete 
most  of  the  activities  within  the  CSDP,  while  also  improving  product  quality  by  conforming  to 
accepted  standards  and  formats  used  by  aircraft  development  organizations.  In  addition,  we 
include  in  the  CDS  an  “encyclopedia”  of  technical  and  management  work  activities  that  span  the 
entire  realm  of  crew  system  development.  This  encyclopedia  is  accessed  directly  through  one  of 
the  CDS  software  tools  and  includes  specific  procedures  that  we  recommend  be  used  to  perform 
each  activity. 

To  present  the  attributes  of  the  new  CDS,  we  employ  a  “walk-through”  method  in  this 
document.  This  walk-through  assumes  that  your  cockpit  design  team  consists  of  individuals  with 
backgrounds  in  all  aspects  of  crew  system  design  such  as  pilot- vehicle  interface,  operational  flight, 
human  factors,  avionics  and  other  subsystem  design  and  systems  engineering.  We  believe  that  this 
approach  is  a  straightforward  way  to  illustrate  the  many  facets  of  the  CDS.  We  have  intentionally 
omitted  a  technical  layout  of  the  CDS  hardware  and  software  architecture  in  order  to  focus  on  the 
content  and  the  envisioned  use  of  the  CCCD  Process  for  cockpit  development  and  upgrade 
programs. 
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Figure  A- 1 .  CCCD  Process  Overview 


To  further  aid  the  reader,  we  periodically  refer  to  a  more  detailed  “map”  of  the  CCCD 
Process,  which  will  be  presented  later.  The  map  identifies  the  key  cockpit  design  activities  that  are 
included  in  the  overview  chart  (Figure  A-l).  The  map  and  this  text  provide  a  concise  and 
consistent  insight  into  the  essential  elements  of  crew  system  design.  While  the  CCCD  procedures 
and  tools  primarily  focus  on  design  concepts,  the  map  extends  beyond  traditional  boundaries  to 
include  planning,  analysis,  design,  development,  and  testing  of  cockpit  crew  systems.  It  is 
important  to  note  that  the  CCCD  Process  Map  integrates  the  CCCD  Process,  procedures,  and 
toolsets  to  improve  the  design  quality  of  both  the  immediate  cockpit  and  the  total  weapon  system. 
The  tools  identified  in  this  guide  are  a  manifestation  of  the  currently  implemented  CCCD  Process, 
and  could  be  replaced  with  other  tools  that  provide  similar  support. 


1 . 2  Background 

In  the  past,  cockpit  design  has  been  somewhat  disjointed,  with  many  different 
organizations,  in  addition  to  the  crew  system  design  group,  vying  for  the  aircrew’s  interaction  with 
their  designs.  Often,  that  was  accomplished  without  a  full  understanding  of:  (1)  the  overall  impact 
of  the  individual  air  vehicle  subsystem  designs  on  cockpit  integration;  (2)  the  established  processes 
imposed  by  contract  requirements  or  internal  policies  (e.g.,  MIL-STDs,  Design  Documents);  or 
(3)  the  combined  impact  of  all  subsystems  upon  aircrew  workload.  More  often  than  not,  the 
cockpit  design  group  was  left  with  the  task  of  fitting  many  independently  designed  panels, 
displays,  and  controls  into  the  available  but  limited  cockpit  volume  (i.e.,  the  major  focus  was  on 
installation,  which  was  necessary  but  detracted  from  effective  design  integration).  The  resulting 
cockpits  could  be  mastered  only  with  great  difficulty,  defied  intuition,  and  failed  to  meet  real-time 
operating  requirements.  As  a  result,  it  was  left  up  to  training  and  the  crew  member’s  ingenuity  to 
make  up  for  limitations  in  the  basic  design.  The  problem  was  compounded  as  cockpits  were 
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upgraded,  because  records  for  the  design  decisions  and  their  underlying  rationale  were  not  kept,  or 
were  not  accessible  for  use  during  the  upgrade.  Consequently,  cockpit  changes  could  have 
violated  earlier  design  assumptions.  Frequently,  cockpit  design  flaws  did  not  become  apparent 
until  flight  test.  This  situation  further  aggravated  cost  and  schedule  if  engineering  changes  were 
approved;  otherwise,  the  flight  crew  simply  learned  to  cope  with  the  problems. 

Designers  soon  came  to  realize  that  a  more  formal,  structured  process  was  needed  to 
improve  cockpit  design  quality.  Recognizing  that  the  Air  Force  lacked  an  advanced  technology 
program  in  the  area,  the  Human  Systems  Center’s  Armstrong  Laboratory  initiated  the  Cockpit 
Automation  Technology  (CAT)  program  in  the  mid-1980s.  The  program  was  renamed  in  the  late 
1980s  as  Crew-Centered  Cockpit  Design,  to  better  reflect  the  aim  of  the  work.  Early  development 
was  nurtured  using  contractor  teams  of  aircraft  manufacturers,  avionics  suppliers,  and  human 
factors  specialists  all  contributing  to  the  formation  of  good  cockpit  design  principles,  with  the 
pilot/crew  aspect  of  design  integration  playing  a  prominent  role.  Five  aircraft  development  prime 
contractors  made  contributions  to  the  program’s  current  technical  state.  To  date,  the  CCCD 
Program  has  produced  an  extensively  documented  but  untested  first  prototype  process  for  cockpit 
design.  In  addition,  this  document  speaks  to  a  future  computer  design  support  system  which  will 
incorporate  much  of  what  we  have  learned  during  this  contract  and  what  the  industry  reviewers 
will  contribute.  Both  phases  are  introduced  in  this  guide.  For  further  information,  a  description  of 
the  evolution  of  the  CDS  toolset  is  provided  in  the  appendix  to  this  document. 

Both  aspects,  the  process  and  the  toolset,  are  now  being  applied  in  a  series  of  field 
demonstration  applications  to  a  variety  of  typical  cockpit  upgrades  and  developments.  To  help 
advance  and  mature  this  technology,  the  advice  of  users  in  the  crew  system  design  community  is 
now  sought  so  that  the  CCCD  Process  and  the  CDS  will  contain  a  sufficient  depth  of  knowledge, 
along  with  the  complete  and  detailed  procedures  for  applying  the  technology  in  a  usable  form  for 
industry  environments. 

Advancing  avionics  now  present  to  designers  the  opportunity  to  consolidate  the  interactions 
among  multiple  onboard  subsystems  into  a  consistent,  integrated  cockpit  environment.  These 
gains,  coupled  with  lessons  learned  from  recent  cockpit  developments,  have  helped  the  cockpit 
design  community  to  better  adhere  to  the  design  practices  that  are  established  in  MEL-H-46855. 
Notably,  designers  of  recent  military  aircraft  cockpits  attempted  to  comply  with  the  specification 
and  used  some  available  (custom)  tools  to  produce  integrated  cockpit  systems.  Industry  designers 
achieved  modest  success  at  formulating  integrated  cockpits.  However,  not  all  recommended 
activities  were  followed,  and  the  necessary  depth  of  analysis,  design,  and  evaluation  was  not 
performed  to  produce  crew-centered  systems  or  to  trace  the  cockpit  design  characteristics  to 
specific  requirements.  In  all  fairness,  industry  had  little  or  no  access  to  a  documented,  verified 
process  and  few  tools  with  which  to  execute  any  process.  Moreover,  time  and  schedule  factors 
precluded  the  full,  iterative  use  of  a  crew-centered  cockpit  design  approach,  denying  the  resulting 
advantage  that  CCCD  offers.  Cost  and  schedule  constraints  are  the  realities  that  will  continue  to 
dominate  what  can  be  achieved  in  crew  system  development.  The  CCCD  Process  and  tools  aim  to 
help  industry  designers  work  more  effectively  within  those  constraints. 

With  the  change  in  emphasis  of  aircraft  systems  and  cockpit  design  activities  came  the 
challenge  to  interact  with  many  different  organizations  to  coordinate  a  consistent,  intuitive,  and 
logical  cockpit.  Integrated  avionics  now  allow  cockpit  functions  to  be  grouped  by  tasks  and  not  by 
individual  systems.  For  example,  mission  avionics  systems  can  gather  target  data  from  various 
sensors,  and  are  able  to  fuse  the  information  into  symbols  on  a  single  display  that  discloses  the 
track’s  identity  and  the  presence  of  radar  or  infrared  radiation.  If  designers  levy  duplicate 
information  requirements  throughout  the  cockpit,  forcing  the  pilot/crew  to  consult  several  displays 
for  dedicated  bits  of  information,  crew  members  will  soon  be  overwhelmed  with  data,  inhibiting 
their  ability  to  extract  the  needed  information.  That  has  been  a  recognized  problem  since  the 
emergence  of  digital  avionics  technology  in  the  mid-to-late  1970s.  Because  the  designers  of 
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individual  subsystems  are  concerned  mainly  with  their  own  systems,  and  only  secondarily  with 
their  impact  on  the  total  cockpit  environment,  cockpit  designers  must  translate  total  crew/mission 
requirements  into  finite,  streamlined  design  features.  Further,  they  must  understand  how  one 
system  affects  another  and  agree  on  a  form  that  complies  with  the  existing  aircraft  design 
specification  process.  This  is  the  challenge  for  today’s  cockpit  designers:  to  communicate, 
coordinate,  and  design  cockpit  systems  that  function  in  support  of  air  crews  performing  essential 
missions. 

From  a  system  design  perspective,  the  Air  Force  acquisition  community  is  moving  to  a  new 
Integrated  Weapon  System  Management  (IWSM)  philosophy.  This  philosophy  relies  on 
Integrated  Product  Development  (IPD),  with  teams  representing  all  of  the  applicable  design 
disciplines.  Based  on  tenets  from  Total  Quality  Management,  the  new  CSDP  approach  also 
borrows  from  the  methods  and  framework  of  concurrent  engineering  and  IPD.  For  years,  industry 
has  employed  a  similar  team  approach  for  cockpit  development,  partly  because  the  cockpit  is  the 
place  where  many  of  the  air  vehicle  subsystems  “come  together”  during  the  process  of  system 
integration.  That  cockpit  integration  process  can  now  be  further  advanced  —  and  hopefully  imple¬ 
mented  for  IWSM  in  major  system  development  —  by  the  structured  and  inspectable  CCCD 
Process  and  its  CDS  family  of  design  support  tools,  once  they  are  proven. 

Giving  further  importance  to  a  human-centered  design  approach  is  the  idea  of  Human 
System  Integration,  which  is  now  mandated  under  the  DOD  5000-series  of  directives  and 
instructions  that  guide  the  major  defense  system  acquisition  programs. 

The  challenge  for  the  CCCD  Program  is  to  mature  the  process,  procedures  and  tools  that 
are  needed  to  sustain  future  mandates  for  proper  crew  system  design,  within  the  context  of  global 
system  design  to  support  mission  objectives.  “Proper”  cockpit  design  means  to  provide:  1)  a 
comprehensive  set  of  understandable  (analysis,  design,  and  evaluation)  activities  that  are  necessary 
and  sufficient  to  produce  a  good  cockpit  design;  2)  a  fully  understood  tradeoff  rationale  leading  to 
the  design;  and  3)  an  understanding  of  why  the  design  is  good.  Due  to  the  fact  that  each  of  these  is 
performed  increasingly  earlier  in  the  weapon  system  design  cycle  (to  produce  the  desired  effect 
upon  the  entire  system),  we  are  further  challenged  to  apply  the  process,  procedures  and  tools  in  a 
quick,  efficient  and  verifiable  manner. 

The  advantage  of  having  a  formal,  structured  process  for  cockpit  design  (and  effective  tools 
for  implementation)  may  be  even  more  important  as  the  defense  industry  is  “downsized,”  to 
augment  the  remaining  workforce  after  the  industry  stabilizes.  The  new  CCCD  Process  and  its 
CDS  Toolset  can  have  a  secondary  value  deriving  from  their  potential  use  in  training  new  crew 
system  designers,  as  some  experienced  designers  approach  retirement. 

Our  crew-centered  design  approach  provides  the  opportunity  for  making  those 
improvements.  By  examining  the  substance  of  the  CCCD  Process,  and  by  deriving  measurable 
“attributes”  during  its  execution,  the  products  of  the  CCCD  Program  can  be  molded  into  a  form 
that  all  can  use.  Although  the  CDS  strives  to  provide  a  standard,  repeatable  crew  system  design 
process,  it  has  yet  to  be  demonstrated,  validated,  or  enhanced.  That  is  the  purpose  and  the  promise 
of  the  current  phase  of  the  CCCD  Program. 

The  CCCD  Process  is  merely  a  reference  model  that  can  be  tailored  and  modified  as  needed 
to  streamline  your  particular  cockpit  development.  Additionally,  it  can  be  used  in  a  variety  of  ways 
and  modified  as  needed  through  careful  program  planning  and  scheduling.  Your  constructive 
suggestions  for  refining  and  extending  the  present  technical  status  of  the  CDS  and  the  CSDP  will 
be  genuinely  valued  inputs  toward  achieving  that  promise. 
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SECTION  2.0:  PROCESS  OVERVIEW 


2.1  Organization 

The  CCCD  Process  organizes  all  activities  necessary  for  cockpit  design  into  five  major 
functional  categories: 

•  Program  Planning 

•  Up-Front  Analysis 

•  Crew  System  Analysis 

•  Crew  System  Design 

•  Crew  System  Evaluation 

These  activities  are  performed  in  parallel  and  iteratively  to  improve  the  quality  of  the  design 
product  (i.e.,  the  cockpit).  In  addition,  the  CDS  includes  specific  procedures  that  guide  the 
performance  of  each  activity,  and  also  has  tools  that  facilitate  the  validation  of  the  cockpit.  Finally, 
the  CDS  supports  the  generation  of  program  deliverables  (e.g.,  specifications  and  contract  CDRL 
data)  which  can  be  implemented  immediately  using  the  CDS  and  delivered  to  the  customer.  These 
deliverables  specify  both  the  actual  design  and  explain  why  the  design  looks  as  it  does. 

The  CSDP’s  distinct  (and  interdependent)  functional  categories  are  purposely  organized 
into  two  phases.  The  first  phase  allows  you  to  perform  functions  of  “Program  Planning  and  Up- 
Front  Analysis”  with  guidance  and  support  from  specified  tools  and  procedures.  These  two 
CSDP  categories  were  omitted  from  earlier  versions  of  the  CSDP.  They  are  done  simultaneously 
so  you  can  plan  the  crew  system  project  as  you  develop  an  understanding  of  its  requirements.  This 
phase  ensures  that  your  plans  meet  the  crew’s  requirements  to  perform  the  mission.  Translating 
the  global  mission  requirements  for  the  weapon  system  into  crew  system  requirements  and  then 
into  engineering  objectives  is  an  important  first  step  within  the  CSDP. 

In  the  second  phase  of  activity,  you  perform  an  organized  set  of  crew  system  analysis, 
design,  and  evaluation  activities,  both  sequentially  and  in  parallel.  The  CDS  contains  software 
tools  and  procedures  to  support  all  aspects  of  this  work.  Importantly,  the  CDS  is  self-contained 
and  small  enough  to  remain  under  the  direct  control  of  the  cockpit  development  team,  which  could 
simplify  or  eliminate  the  need  for  interfacing  with  another  support  department.  The  elements  of 
crew  protection  (e.g.,  life  support  equipment,  escape  systems,  and  environmental  control)  are 
critical  to  cockpit  development  projects,  especially  for  combat  systems  that  employ  ejection  seats  or 
escape  capsules;  thus,  the  CSDP  provides  for  their  consideration.  Due  to  the  fact  that  escape  and 
life  support  technologies  are  currently  being  worked  in  separate  advanced  projects  within  the  DoD, 
the  CDS  Process  calls  for  an  analytical  look  at  these  systems  as  a  means  of  revealing  the  important 
cockpit  integration  factors,  but  does  not  delve  into  further  detail. 

As  illustrated  in  the  maps  and  their  accompanying  CSDP  tools  and  procedures  that  follow, 
many  of  the  activities  within  the  second  phase  need  to  be  performed  interactively  with  other 
activities  and  iteratively  within  the  development  cycle,  .a  order  for  the  process  to  truly  improve 
quality.  Each  step  of  the  way,  the  CDS  recommends  the  appropriate  action  with  respect  to  the  plan 
you  have  developed,  through  text  and  graphic  aids.  In  addition,  the  timing  of  all  activities  and 
products  adhere  to  vour  schedule.  Tools  and  procedures  are  also  being  developed  that  will  allow 
for  quick  delivery  of  intermediate  and  final  products. 
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The  key  to  using  the  CCCD  procedures  and  tools,  and  also  to  understanding  the  timing  of 
activities,  resides  in  the  ability  to  “see”  how  all  of  the  parts  fit  together.  In  order  to  unlock  these 
aspects,  we  developed  a  “map”  of  the  CCCD  Process  and  a  narrative  guide  that  explains  the  map. 


2.2  CCCD  Process  Map 


The  CCCD  Process  Map  (referred  to  below  as  the  “Map”)  depicts  the  “mainline”  activities 
performed  when  developing  a  cockpit  design.  Of  course,  there  are  many  other  potential  activities 
that  may  need  to  be  performed,  depending  upon  the  objectives  of  your  particular  program.  Within 
the  Map,  we  embedded  “avenues  of  access”  for  those  events,  at  appropriate  stages. 

The  Map  is  a  computer-interactive  way  to  guide  you  along  the  development  of  your 
cockpit,  throughout  the  life  cycle  of  your  program.  As  such,  it  provides  you  and  your  design  team 
with  the  ability  to  understand  the  issues  surrounding  immediate  tasks,  and  includes  the  perspective 
to  see  how  that  activity  fits  within  the  entire  design  approach.  The  Map  comes  in  two  forms,  a 
“micro  look,”  which  shows  procedural  detail  of  an  activity,  and  a  “macro  look,”  which  shows  an 
activity  in  relation  to  all  others.  Macro  maps  contain  useful  information  about  both  input/output 
requirements  and  activity  status.  Together,  micro  and  macro  views  of  the  CCCD  Process  Map 
offer  guidance  about  the  CSDP  activity  order,  the  scheduling,  the  product  technical  development, 
and  the  overall  timing  of  program  activities. 

At  first,  the  Map  may  deceive  you  into  thinking  that  the  process  is  too  large  to  be 
accomplished.  Although  it  shows  a  considerable  number  of  activities,  the  provision  of  procedures 
for  performing  the  individual  CSDP  activities,  and  the  availability  of  the  CDS  tools,  will  allow  you 
to  accomplish  more  activities  with  little  or  no  increase  in  time  or  cost,  when  compared  to  past 
cockpit  development  programs.  This  is  due  in  part  to  the  elimination  of  input/output  formatting 
problems  for  much  of  the  design  data,  which  can  be  automated  within  the  CDS.  When  fully 
developed  and  proven  in  applications,  the  CDS  should  enable  design  teams  to  produce  a  high 
quality  of  cockpit  design,  while  tracing  requirements  flowdown  and  design  decisions  throughout 
development. 


Of  the  many  activities  that  need  to  take  place,  some  must  be  re-accomplished  on  an  iterative 
basis.  This  is  necessary,  in  the  sense  that  one  acquires  greater  knowledge  with  each  pass  (adding 
value  to  later  dependent  activities).  It  is  also  desired,  in  the  sense  that  all  final  products  are  made 
better  by  a  continual  cycle  of  inspection,  analysis,  and  change.  The  CCCD  Process  relies  upon  the 
design  iteration  (through  several  cycles  of  the  prototyping,  analysis,  and  simulation  in  support  of 
tradeoff  design  activities).  The  process  also  relies  upon  activity  interaction  (between  analysis, 
design  and  evaluation  activities)  to  ensure  quality  during  the  process. 


The  overview  CCCD  Process  Map  (Figure  A-2)  illustrates  the  interactions  among  activities, 
and  invokes  the  iterative  nature  of  the  cockpit  design  process.  In  addition,  the  map  shows  both 
sequential  and  parallel  activities  during  the  development  cycle.  In  this  way,  the  CCCD  Process 
reflects  and  builds  upon  current  industry  practices.  Within  the  Map  are  the  five  functional 
categories  that  were  listed  previously,  Program  Planning,  Up-Front  Analysis,  Crew  System 
Analysis,  Crew  System  Design,  and  Crew  System  Evaluation).  These  categories  are  shape  coded 
as  follows: 

Legend: 

Program  Planning  Activities 


Up-Front  Analysis  Activities 
(o°»»»)  Crew  System  Analysis  Activities 


Q 

<3> 


Crew  System  Design  Activities 
Crew  System  Evaluation  Activities 
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In  addition  to  the  shapes,  you  will  notice  that  in  the  detailed  process  maps  (Figures  A-5  to 
A-8),  a  certain  number  of  activities  are  shaded  darker  than  the  others.  This  notation  is  used  to 
identify  critically  important  documentation  prepared  during  the  process.  Many  of  these  reports  and 
documents  are  common  CDRL  requirements,  while  others  are  significant  internal  communications. 
A  major  aim  of  the  CCCD  Process,  Procedures  and  Tools  is  to  support  the  development  of 
required  documentation. 

The  Map  is  a  kind  of  process  flow  chart,  where  time  generally  “flows”  toward  the  right 
side  of  the  chart  (reflecting  the  availability  of  interim  results  as  the  design  becomes  refined  over 
time).  As  will  be  seen  later  when  parts  of  the  chart  are  detailed,  the  actual  development  of  the 
design  (the  second  phase  noted  above,  distinct  from  the  Up-Front  Analysis  and  Program  Planning 
phases)  is  represented  by  separate  “threads”  for  Analysis,  Design,  and  Evaluation  work,  each  of 
which  is  stacked  vertically  and  shown  as  a  horizontal  set  of  activities  on  the  chart. 

The  Map  is  also  implemented  as  software  within  the  CDS  and  is  intended  to  be  one  of  the 
on-line  planning  tools  to  help  crew  system  managers  and  engineers  make  informed  choices  about 
the  development’s  progress. 


2.2.1  Discussion  of  Map  Phases 

Throughout  the  Map,  you  may  notice  our  attempt  to  force  tradeoffs,  results,  and 
conclusions.  We  believe  that  decisions  should  be  made  based  on  crew  and  mission  requirements, 
and  not  primarily  from  single  interest  or  “group  think”  sources.  The  CCCD  ideology  provides  an 
inherent  way  to  trace  your  design  decisions,  using  both  qualitative  and  quantitative  results,  in  a 
manner  that  promotes  both  efficiency  and  design  effectiveness  while  still  utilizing  operator  opinion. 
The  Map  phases  discussed  below  follow  the  two-phased  structure  noted  above. 


2. 2.1.1  First  Phase 

The  initial  phase  of  the  Map  assists  in  performing  two  critical  categories  of  activities  prior 
to  developing  your  cockpit: 

•  Program  Planning  (ellipses),  in  which  the  necessary  activities,  schedule,  and  cost 
estimates  are  defined,  to  link  the  cockpit  design  to  requirements;  and, 

•  Up-Front  Analysis  (rounded  boxes),  in  which  the  cockpit  requirements  are  derived  from 
upper-tier  requirements. 

Other  products  from  this  section  yield  initial  information  for  other  activities  (such  as 
notional  cockpit  design,  system  drivers,  mission  profile  data,  etc.).  Each  individual  activity  is 
discussed  within  its  respective  functional  category.  The  key  output  products  from  activities  within 
the  first  phase  are  a  Design  Requirements  Document,  the  Initial  Crew  System  Specification,  and  a 
Program  Plan. 


2.2. 1.2  Second  Phase 

The  final  phase  of  the  Map  is  designed  to  assist  you  in  performing  three  critical  categories 
of  cockpit  design  activity  and  two  value-added  improvements  to  developing  the  cockpit  as  follows: 

•  Crew  System  Analysis; 
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•  Crew  System  Design;  and, 

•  Crew  System  Evaluation. 

Value-added  improvements  are  represented  on  the  Map  by  the  interactions  among  the 
activities  (shown  by  arrows)  and  by  design  iterations  (shown  by  repetition  of  design  activities), 
which  are  performed  as  many  times  as  possible  within  your  timetable.  The  highlights  for  each  of 
the  three  types  of  activity  within  the  second  phase  are  summarized  below,  in  turn. 

The  Crew  System  Analysis  section  (octagons)  of  the  Map  represents  two  vital  functions  in 
the  development  of  your  cockpit: 

•  Derivation  and  verification  of  crew  requirements;  and, 

•  Definition  of  the  operating  limits  for  crew  and  system. 

The  principal  output  products  from  the  Crew  System  Analysis  activities  are  the  Mission 
Profile,  Mission  Scenario,  Functional  Flow,  Action/Information  Requirements,  Function 
Allocation  Trade,  Task  Timeline,  Task  Workload  Analysis,  and  updates  of  the  Cockpit  System 
Specification,  and  the  Cockpit  Traceability  Report.  The  outputs  can  be  in  paper  or  electronic  form, 
and  can  include  text  files,  charts,  graphics,  and  computer  data  files. 

The  Crew  System  Design  section  (rectangles)  represents  two  essential  functions  in  the 
design  of  your  cockpit: 

•  Integration  of  all  information  about  your  system  (for  defining  the  cockpit  design);  and, 

•  Production  of  the  system  documentation  of  that  design. 

The  principal  output  products  from  the  Crew  System  Design  activities  include  the  Cockpit 
Mechanization  Document,  updates  to  the  Cockpit  System  Specification,  and  the  Cockpit 
Traceability  Report.  Again,  the  outputs  can  be  in  electronic  form  or  in  hard  copy. 

The  Crew  System  Evaluation  section  (hexagons)  will  help  guide  you  in  performing  what  is 
perhaps  the  most  important  function  associated  with  your  cockpit  design:  cockpit  design 
verification.  In  this  section,  your  design  is  verified  in  terms  of  its  crew/mission  requirements, 
acceptance,  implementation,  and  intuitive  appeal.  In  addition  to  evidencing  its  “market  appeal,” 
data  is  developed  to  convince  your  customer,  SPO  and/or  the  user  commands  that  your  design 
meets  or  exceeds  requirements. 

The  principal  output  products  from  the  Crew  System  Evaluation  activities  include  the 
Dynamic  Simulation  Plan,  Mockup  Evaluation  Plan  and  Report,  Rapid  Prototyping  Evaluation 
Plans  and  Reports,  Part-Task  Simulation  Plans  and  Reports,  Full  Mission  Simulation  Test  Plans 
and  Reports,  Flight  Test  Plan,  and  individual  Flight  Test  Plans  and  Reports. 

All  of  these  output  products  (in  both  phases)  take  on  added  importance  because  they  are  the 
observable  marks  of  your  technical  progress.  To  the  extent  that  they  can  be  easily  retrieved  and 
inspected,  they  can  be  used  in  the  sustaining  engineering  phase  of  the  system  life  cycle  for 
managing  changes.  During  this  later  phase,  it  is  unlikely  that  personnel  from  the  original  cockpit 
design  team  will  be  available  for  guiding  these  changes. 

The  Map  also  shows  how  and  when  the  interaction  of  activities  needs  to  occur  in  order  to 
better  perform  all  activities.  In  addition  to  the  lines  of  communication  shown,  each  procedure 
promotes  the  use  of  information  from  other  products.  The  final  feature  of  the  Map  is  its  ability  to 
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show  the  nature  and  timing  of  design  iteration.  The  key  to  improving  cockpit  design  quality 
resides  in  one’s  ability  to  redesign,  and  effectively  us£  ail  available  information  from  analysis, 
design,  and  evaluation.  When  fully  exploited,  the  combined  information  produced  from  these 
activities  can  have  a  profound  effect  on  your  design.  The  keys  to  help  you  gain  that  advantage  are 
the  availability  of  the  information  (assured  by  performing  the  CSDP  using  the  recommended 
procedures)  and  the  ability  to  quickly  access  the  data  when  needed  (assured  by  the  use  of  the  CDS 
data  management  software). 


SECTION  3.0:  CDS  IMPLEMENTATION 


3 . 1  CDS  Implementation 

The  CCCD  Process  is  represented  in  software  that  runs  on  the  CDS.  The  implementation 
is  in  the  form  of  a  “Guidebook”  that  is  analogous  to  the  CCCD  Process  Map  in  this  document,  but 
that  also  provides  all  the  detailed  explanations,  experiences,  guidelines,  and  requirements  needed  to 
perform  all  phases  of  crew  system  design. 

An  earlier  version  of  the  CCCD  Process  was  documented  in  several  cumbersome  paper 
reports,  separately  tailored  for  each  of  the  four  classic  phases  of  weapon  system  acquisition.  That 
was  useful  as  a  reference  and  to  place  bounds  on  the  actual  CCCD  Process,  but  clearly  the  hard 
copy  representation  was  not  streamlined  for  implementation.  Taking  the  next  step  toward  that 
implementation,  and  to  permit  a  better  understanding  of  the  crew-centered  approach,  we 
“repackaged”  the  hard  copy  design  process  to  the  simpler  form  described  herein.  In  the  sections  to 
follow,  the  new  CCCD  Process  is  pictorially  and  textually  presented  in  a  form  that  will  aid  the 
designer’s  appreciation  of  the  interactions  among  the  many  CSDP  activities.  The  new  crew- 
centered  design  process  is,  itself,  implemented  as  interactive  software,  which  allows  the  crew 
system  designer  to:  (1)  inquire  about  an  activity;  (2)  directly  perform  an  activity;  and  (3)  plan  and 
manage  single  or  multiple  sets  of  activities  through  a  collection  of  (prompted)  procedures  and 
tools. 


3.1.1  Activities 


An  activity  is  the  lowest  level  of  coordinated  effort  needed  to  gain  a  specific  understanding 
of  an  event,  or  a  series  of  events.  For  example,  uus  definition  applies  to  information  and  control 
requirements  analysis,  to  the  design  of  a  particular  display  format,  such  as  a  Tactical  Situation 
Display  format,  and  to  the  preparation  of  a  test  plan  for  the  initial  part-task  simulation  evaluation. 
In  each  case,  there  are  individual  CSDP  procedures  that  should  be  accomplished  to  conclu^?  the 
activity  with  an  output  product,  such  as  the  principal  outputs  identified  above. 


3.1.2  Procedures 


A  procedure  is  a  set  of  individual  actions  that,  collectively,  produces  a  repeatable  outcome. 
We  prescribe  standardized  sets  of  valid  actions  for  each  activity,  which  can  be  implemented  alone 
or  through  the  use  of  an  integrated  toolset  such  as  what  exists  within  the  CDS.  Simply  stated,  we 
recommend  and  describe  (in  a  sequential,  detailed  electronic  format  on  the  CDS)  which  analytical 
or  design  or  evaluative  factors  to  apply;  which  activities  or  database  inputs  are  necessary  prior  to 
action;  what  actions  are  required;  how  to  derive  conclusions  and  results  from  each  action;  and 
useful  output  in  terms  of  products  that  can  “stand  alone”  or  support  other  activities  as  inputs.  The 
product  orientation  for  the  CSDP  activities  and  procedures  serves  the  four  purposes:  (1)  it  clarifies 
the  technical  tasking;  (2)  it  supports  management  by  providing  visibility  into  the  progress  of  the 
design;  (3)  it  directly  provides  a  means  to  record  the  technical  rationale  and  traceability  of  the 
cockpit  design;  and  (4)  it  permits  the  potential  re-use  of  CSDP  data  by  other  Product  Development 
Teams. 
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3.1.3  Tools 


A  tool  is  an  instrument  which,  under  the  control  of  a  skilled  user,  can  aid  in  performing 
work  activities.  The  CDS  contains  several  software  tools  that  were  specifically  made  to  support 
most  of  the  activities  that  are  shown  on  the  Map.  When  applied  to  an  activity,  the  user  benefits 
most  from  cost  and  schedule  savings.  Once  fully  developed  and  tested,  the  CDS  tools  will  be  a 
convenient  way  to  carry  out  the  activities  specified  by  the  CCCD  Process.  It  is  important  to  note 
that  the  CDS  is  not  intended  to  replace  the  designer,  or  to  make  a  novice  designer  into  a  skilled 
design  professional.  The  CDS  cannot  supply  the  creativity  or  ability  of  an  individual  to  perform 
this  specialized  work.  As  a  design  support  system,  however,  it  can  provide  guidance  to  the 
cockpit  design  team,  markedly  improve  communications  within  the  integrated  product  team,  help 
“prove  out”  the  design  at  an  earlier  stage  in  system  development  (i.e.,  promote  efficiency),  and 
help  to  attain  a  total  quality  design  product  from  both  crew  and  mission  standpoints. 


3.2  Management  of  CDS 

The  CDS  is  controlled  through  a  software  interface  tool  known  as  the  Design  Traceability 
Manager,  or  DTM.  DTM  provides  a  standard  interface  for  all  other  tools  within  the  CDS.  It 
supports  project  management  and  scheduling,  guides  the  design  team  in  the  use  of  CDS,  records 
day-to-day  activities,  maintains  design  and  decision  traceability,  and  accommodates  multiple  users 
like  those  in  an  Integrated  Product  Development  environment.  These  are  just  examples  of  the 
depth  behind  this  management  system.  In  essence,  the  DTM  can  be  viewed  as  a  software 
"executive"  for  the  CDS. 

A  simple,  consistent,  and  flexible  windowed  environment  is  at  the  core  of  the  DTM.  It 
allows  straightforward  management  of  many  of  the  details  within  your  responsibility.  In  a  typical 
work  session,  you  use  a  mouse  to  select  one  of  the  following  five  functional  areas  of  the  "top- 
level"  page  (Figure  A-3): 

•  Main  Menu:  Project  Management  Functions,  DTM  Utilities,  and  Tool  Execution 

•  Current  Status:  Current  Status  of  User,  Project  Information 

•  Interactive  Data  Entry  Forms:  Product  Traceability  Functions,  Lessons  Learned 

•  Navigation  Buttons:  Individual  and  Multiple  CSDP  Activity  Control 

•  Current  Activity:  Work  Space  of  Activities  and  Procedures  (Scrolling) 

Through  DTM,  you  can  learn  who  is  responsible  for  an  activity,  how  to  accomplish  the 
activity,  what  information  or  database  to  use,  when  your  results  will  be  needed,  what  type  of 
results  are  needed,  and  where  they  will  be  used  within  the  project.  In  that  way,  you  have  visibility 
into  how  your  work  products  fit  into  the  overall  scheme  of  development.  You  can  trace  past 
activities,  log  new  ones  into  the  system,  and  call  up  a  CCCD  Process  Map  that  is  specifically 
tailored  to  your  project,  to  find  out  more  about  individual  pieces  or  to  determine  progress  within 
the  process. 

At  this  point,  it  is  useful  to  note  that  the  CCCD  Process  is  not  a  rigid  way  to  perform  work 
that  will  be  levied  on  you.  Rather,  it  is  a  reference  model  that  can  be  tailored  and  modified  as 
needed  to  streamline  your  development.  Because  it  resides  within  a  computer  implementation,  it 
can  be  used  in  a  variety  of  ways  and  modified  as  needed  (by  those  having  such  authorization 
privileges). 
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Project  ActivltiM  Flit  Report  CDS  Tools  Reference  Kits  Administrator  Help 


Currant  Status 

Current  Activity 

|  A3 . 4  Per fora  Mission  Profile  Analysis 

Project  1  Testing 

A3 .1 

Perform  Life  Support  Analyses 

Context  | H2SM0  CFGO 

A3. 2 

Perform  Escape  Analyses 

A3. 3 

Perfora  Reach  Analysis 

A3. 4 

Perform  Mission  Profile  Analysis 

A3. 8 

Class  I  Analysis 

A3. 6 

Perform  Mission  Scenario  Analysis 

A3. 7 

Complete  Scenario  Analysis  Report 

Logbook  1 

A3. 8 

Perfora  Mission  Phases/Functional  Flow  Analysis 

Product  Traceability  f 

A3. 9 

Complete  Functional  Flow  Analysis  Report 

Loaaona  Learned  j 

File  Traceability  | 

Current  Procedure  j 

* 

P3.4-1 

Brainalora  the  ■iasien  events  which  need  to  take  place  wit hit 

P3.4-2 

Layout  routes  graphically 

Navigation  Buttons 

Activity  Information  J 

P3.4-3 

Add  environmental  and  threat  events  such  as  weather.  SAM's,  t 

II 

II 

— - = - 

1  Upper  Level 

P3.4-5 

Examine  profiles  and  determine  HOF 'a.  HOU's  ft  NOP 'a  associate 

P3.4-6 

Create  a  PTR  for  the  Mission  ProTAln  Analysis 

Procedure  Inforastlon  J 

/ 

^Currently  accessing  database 


Figure  A-3.  Example  DTM  Page 

The  crew  systems  engineer  can  select  specific  sections  of  the  process  to  investigate.  In  an 
interactive  "session"  with  DTM,  the  user  can  progress  through  finer  levels  of  the  process  until  the 
desired  information  is  located.  This  indentured  presentation  of  topics  permits  rapid  navigation  to 
a  precise  area  of  interest. 

Once  an  engineer  selects  an  avenue  of  investigation,  the  DTM  guide  reveals  formal  proce- 
duralized  activities,  likely  approaches,  lessons  learned,  and  pertinent  experiences.  Design 
constraints  and  interdependencies  will  also  be  displayed.  This  procedure  page  (Figure  A-4) 
contains  summary  information  that  will  be  vital  to  the  swift  completion  of  the  referenced  activity. 
We  also  provide  a  detailed  method  for  accomplishing  each  activity.  Further,  we  recognize  that 
there  may  be  other  ways  of  performing  the  same  work.  Our  procedures  merely  reflect  proven 
methods  to  complete  all  activities.  The  example  procedure  page  is  shown  for  the  activity  of 
“Perform  Functional  Flow  Analysis.”  This  information  includes:  (1)  the  functional  category  of 
the  activity,  i.e.,  Up-Front  Analysis,  Program  Planning,  Analysis,  Design,  or  Evaluation;  (2)  the 
actions  required  to  perform  the  activity;  and  (3)  detailed  sub-actions  that  describe  predecessor 
activities,  individual  procedures,  management  considerations,  output  requirements,  database 
references,  and  recommended  tools  to  perform  the  work. 


3.3  CDS  Tools 

In  order  for  the  design  team  to  accomplish  its  gamut  of  activities,  such  as  performing 
needed  analyses,  formulating  and  deciding  among  tradeoffs,  producing  data  and  physical  products. 
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Perform  Functional  Flow  Analysis 

Description:  Functional  Flow  Analysis 

Functional  Flow  Analysis  is  used  to  establish  the  flow  of  critical  mission  segments  and  to  provide  a  vehicle  lor  breaking  down  the  critical  mission  events  to  the  task 
level.  The  analysis  uses  critical  mission  segment  Level  I.  II  and  III  block  diagrams  as  a  means  for  producing  the  functional  flows.  The  following  desenbes  me  content 
associated  with  each  of  the  block,  diagram  levels. 

Level  I  block  diagrams  will  be  used  to  develop  the  overall  flow  of  the  composite  mission  from  start  to  finish.  The  mission  will  be  broken  down  to  me  level  of  cmical 
mission  segments  based  on  a  specific  or  composite  mission.  Mission  phases  will  define  each  mission  segment  to  be  considered  for  subsequent  analysis  (e  g., 
ingress,  target  area  operations,  egress,  etc.). 

The  purpose  of  the  Level  II  block  diagrams  is  to  establish  the  flow  of  me  mission  at  the  gross  task  level  and  will  be  directly  based  on  the  critical  mission  segments 
defined  within  the  Level  I  block  diagrams.  Level  II  block  diagrams  define  the  major  functional  tasks  to  be  performed  within  each  of  the  specified  cntical  mission 
segments. 

The  Level  III  block  diagrams  will  be  developed  by  further  detailing  the  Level  II  block  diagrams  and  performing  an  initial  system/pilot  task  allocation  based  on  a 
preliminary  assessment  of  me  repeatability  of  the  task  and  me  required  accuracy  of  performance.  The  Level  lit  bloat  diagrams  define  the  critical  mission  segments  to 
the  level  of  me  operator  task. 

PtttlflCMMtl&V. 

in  order  to  proceed  with  the  Functional  Flow  Analysis  you  will  need  to  have  some  details  about  the  order  of  me  mission  events  and  the  type  of  events  which 

need  to  take  place.  A  mission  scenario  and  mission  profile  will  supply  you  with  mis  type  of  information. 

Procedurals): 

1)  Break  down  the  missionfs)  Into  distinct  phases  of  operation  based  upon  previous,  similar  missions  end  the  introduction  of  new  technologies  and/or 
tactics  (block  I  diagrams) 

•  Look  at  the  logical  flow  of  events  contained  in  me  mission  profile  and  scenario  and  construct  a  functional  segment  breakout  of  each  mission.  This  typically  is  terms 
such  as  pre-flight,  takeoff,  enroute  cruise,  etc. 

•  Assess  similar  type  mission  functional  flows  from  the  mission  functional  flow  database  to  verify  your  terminology  and  placement  of  mission  segments. 

•  Create  a  numbering  system  of  the  block  diagram  entries  in  order  to  provide  traceability  between  the  lower  and  higher  level  functions  and  between  functions  at  the 

same  level.  This  same  numbering  system  should  be  used  throughout  all  other  crew  system  analysis  activities  for  consistency  purposes. 

Typically  me  numbering  system  is  1.0  for  pre-flight  segment,  2.0  for  takeoff  segment,  3.0  tor  enroute  cruise  segment,  etc.  Then  each  segment  will  be  broken 
down  further  for  each  level  of  information.  An  example  might  be  1.1  is  the  pre-flight  checklist  accomplishment  and  1.1.1  it  check  brake  pressure,  which  is  the  first  item 
of  me  pre-flight  check  list.  Presentation  of  me  mission  data  in  the  block  level  diagram  (graphic)  format  permits  the  user  to  grasp  the  mission  at  each  ol  the  levels  of 
detail  available,  and  to  understand  me  task  requirements  associated  with  each  mission  phase.  It  it  typically  more  effective  than  a  textual  tabular  format  of  sequential 
indentured  tasks. 

•  Lay  out  the  sequential  segments  tor  the  missk>n(s)  being  analyzed 

2)  Provide  e  written  description  ol  the  events  contained  within  each  phase 

•  Put  together  a  written  account  of  what  major  segments  will  be  taking  place  during  each  mission.  Any  insight  into  the  particular  cockpit  requirements  of  those  phases 
should  be  documented  as  welt  for  future  use. 

3)  Break  down  distinct  mission  phases  Into  functional  ecttvltlee  performed  by  each  crew  member  for  each  segment  (Mock  If  diagram) 

•  Look  at  the  logical  flow  of  events  contained  in  the  block  I  diagram  of  the  mission  and  decompose  a  functional  activity  breakout  of  each  segment.  This  typically  is 
terms  such  as:  perform  fence  check,  update  navigation  system,  contact  with  tanker,  etc. 

•  Assess  similar  type  mission  functional  flows  from  the  mission  functional  flow  database  to  verify  your  terminology  and  placement  of  functional  activities. 

•  Continue  with  your  numbering  system  of  the  block  diagram  entries. 

•  Lay  out  the  sequential  functions  within  each  segment  tor  the  mission(s)  being  analyzed 

4)  Provide  a  written  description  of  the  events  contained  within  each  functional  activity 

•  Put  together  a  written  account  ot  what  major  functional  activities  will  be  taking  place  during  each  mission  segment.  Any  insight  into  the  particular  cockpit 
requirements  for  each  functional  activity  should  be  documented  as  wefl  tor  future  use. 

5)  Break  down  each  functional  event  into  distinct  tasks  performed  in  order  to  complete  the  functional  flow  breakdown  to  Ha  lowest  level  (Mock  M  diagrams) 

•  Look  at  the  logical  flow  of  events  contained  in  the  block  II  diagram*  of  the  mission  and  decompose  a  distinct  task  breakout  ot  each  functional  activity.  This  typically 
is  terms  such  as:  acquire  target  visually,  ftra  AIM-9L,  observe  target  destroyed,  etc.  This  can  be  a*  many  tasks  or  sub  tasks  as  necessary  to  decompose  the  task 
down  to  a  single  action  (even  when  more  than  one  sensory  condition  applies),  (block  III  or  mors  diagram) 

•  Assess  similar  type  mission  functional  flows  from  the  mission  functional  flow  database  to  verify  your  terminology  and  placement  of  mission  tasks. 

•  Continue  with  your  numbering  system  of  the  block  diagram  entries. 

•  Lay  out  the  sequential  functions  within  each  segment  tor  the  missions)  being  analyzed 

PnySipMifl- 

Upon  concluding  your  function  flow  breakout,  you  wIN  have  the  documentation  for  developing  aU  levels  of  flow  throughout  the  mission.  This  data  will  be  used  to 
graphically  produce  the  mission  flows  whan  you  produce  s  report. 

Each  ol  the  Level  III  Blocks  of  information  will  be  usad  to  provide  the  basis  for  the  action/information  requirements  analysis  and  the  task  timeline  and 
task/wo rtdoad  analyses. 

Successor^): 

The  information  from  this  analysis  will  be  gathered  for  a  report  in  which  the  mission  will  be  portrayed  in  a  graphical  flow  format  The  information  will  also  be  used 
to  perform  action/information  requirement*  analysis  and  the  task  timeline  and  taak/woddoad  analyses. 

RflcomiMndfldJgglttli 

The  Functional  Flow  Analysis  tool  (that  is  currently  Concept  Mapper  but  Design  3.0  is  also  a  good  choice  -  more  flexible)  should  be  usad  to  develop  the 
information.  The  lowest  level  block  textual  data  will  also  become  the  basis  for  a  subsequent  analyses  and  therefore  should  be  entered  into  TMT  as  is.  including  the 
numbering  system. 

Recommended  Reference  DataMseia); 

The  yet  to  be  developed  Functional  Flow  database  w»  need  to  be  accessed  to  look  at  previously  accomplished  functional  flow*  of  similar  mission  and  crew  size 
requirements. 


Figure  A-4.  Example  DTM  Procedure  Page 
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and  integrating  (electronically  and  practically)  the  large  amount  of  cockpit-related  information 
within  the  applications  project,  we  are  developing  tools  and  databases  to  support  these  necessary 
functions. 

Most  CDS  activities  and  products  use  a  database  or  associated  software  tool  to  expedite 
task  accomplishment.  We  consider  an  expedited  task  to  be  one  where  the  engineer  considers 
multiple  information  sources,  uses  design  data  from  validated  concepts  proven  in  related 
applications,  and  meets  time/schedule  deadlines  to  produce  an  effective  and  operable  cockpit 
design. 


3.4  CDS  Usage 

To  fully  appreciate  the  value  of  the  CCCD  Process  and  the  CDS  Process  Map,  the 
following  material  will  step  you  through  a  generic  cockpit  design  example.  It  was  prepared  in  the 
form  of  a  “users  guide”  to  promote  understanding  within  the  cockpit  design  community.  Our 
intent  is  for  you  to  focus  on  process  requirements  as  you  critique  the  material. 

Throughout  this  description,  we  identify  the  databases  and  tool  attributes  that  are  required 
to  accomplish  the  activities  and  to  produce  observable  products.  Currently,  some  of  these  tools  are 
prototypes,  some  are  beginning  development,  while  others  are  being  derived  from  design 
requirements.  An  initial  set  of  CCCD  tools  from  another  earlier  development  contract  (CAT)  was 
re-hosted  and  evaluated,  leading  to  the  present  status  of  the  CDS.  We  include  below  a  description 
of  the  present  and  proposed  enhanced  CDS  toolset  to  illustrate  the  premise  by  which  the  CCCD 
Process  is  being  implemented.  As  you  review  this  guide,  please  keep  in  mind  that  we  need  your 
expertise  to  identify  any  appropriate  tool  modifications  and  new  requirements  that  will  enhance  the 
CCCD  Process,  procedures,  and  tools  so  that  they  can  meet  your  needs  as  crew  system 
developers.  As  noted  earlier,  a  historical  development  of  the  CDS  toolset,  as  it  evolved  and  where 
it  is  heading,  is  described  in  the  appendix  to  this  document. 

As  a  notational  convenience,  we  represent  below  the  updates  and  refinements  to  earlier 
design  information  during  the  application  of  the  CCCD  Process  in  italics,  the  use  of  specific  CDS 
software  tools  in  bold  print,  and  the  specific  observable  output  products  from  CSDP  activities  are 
underlined. 

Each  of  the  activities  marked  with  bullets  (•)  in  the  following  paragraphs  coincide  with 
those  comparable  activities  on  the  accompanying  CCCD  Process  Map. 


START  OF  PROGRAM 

You  begin  by  addressing  USAF  requirements  documents  that  are  typically  used  to  initiate  a 
new  aircraft  program.  Similar  requirements  documents  are  used  for  cockpit  modification 
programs,  both  during  the  development  and  the  sustaining  engineering  program  phases.  These 
documents  include  a  System  Operational  Requirements  Document  (SORD),  Statement  Of  Need 
(SON),  initial  Prime  Item  Development  Specification  (PIDS),  and  Threat  Assessment  Report 
(TAR).  Often,  these  formal  requirements  documents  are  made  available  to  the  industry  Program 
Manager  by  the  SPO  as  an  addendum  to  the  aircraft  development  contract. 
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PROGRAM  PLANNING  AND  UP-FRONT  ANALYSIS 

The  following  detailed  map  (Figure  A-5)  of  Program  Planning  and  Up-Front  Analysis  will 
help  guide  you  through  the  next  two  sections  of  this  document.  The  activities  are  closely 
dependent  upon  each  other  to  determine  the  direction  of  your  cockpit. 

•  Examine  SORD 

•  Examine  SON 

•  Examine  PIDS 

•  Examine  TAR 

•  Identify  Known  Similar  Cockpits  and  Technology 

As  always,  the  first  step  is  to  examine  Government-furnished  documentation  to  understand 
what  direction  to  take  the  cockpit.  The  documents  will  already  have  been  summarized  (possibly  by 
someone  in  the  group)  to  include  the  key  design  elements  for  the  cockpit  system.  This  information 
is  entered  into  the  program  documentation  database  within  the  CDS.  All  members  of  the  cockpit 
integrated  product  team  should  glean  what  they  can  from  the  stated  requirements,  and  contrast  the 
system  demands  against  background  information  that  they  bring  from  their  individual  areas  of 
expertise.  The  design  team  has  access  to  several  information  sources  within  the  CDS.  Several 
databases  such  as  cockpit  configurations,  lessons-learned,  projected  cockpit  technology, 
mission/task  analysis,  and  cockpit-related  military  standards  and  specifications  are  also  planned  for 
implementation.  Ideally,  all  project-specific  information  files  would  reside  within  a  relational 
database  program  of  Database  Management  Software  (DBMS)  that  is  accessible  through  the  CDS’ 
Design  Traceability  Manager  (DTM).  Of  course,  each  new  development  will  have  other 
information  that  will  not  reside  in  the  CDS  at  the  outset  of  the  project  and  will  have  to  be 
assembled.  The  crew  system  design  manager  then  initiates  the  cockpit  project  through  the  DTM, 
creating  a  place  for  future  electronic  documents. 

•  Evaluate  Results  of  System  Requirements  Documentation 

The  next  step  is  to  use  an  “up  front”  analysis  technique  to  help  the  team  translate  system 
requirements  to  cockpit  requirements,  and  then  to  engineering  design  objectives.  The  DBMS  in 
CDS  readies  each  document  for  critical  on-line  examination.  From  there,  you  pare  down  the 
information  into  priority  crew  requirements  and  mission  requirements ,  which  can  be  traded  against 
each  other.  This  is  done  to  determine  which  requirements  should  have  the  highest  weighting  to 
assist  you  in  deciding  the  direction  of  your  cockpit  design  approach.  The  lessons-learned  database 
also  provides  knowledge  from  previous  programs  to  assist  you  in  prioritizing  the  requirements. 
However,  as  in  current  practice,  it  will  be  a  judgment,  based  upon  the  experience  of  your  design 
team  and  the  project  leaders,  which  ultimately  guides  your  approach. 

•  Determine  System  Drivers 

•  Define  Problem  Statement 

•  Define  Cockpit  Philosophy 

The  final,  prioritized  requirements  will  be  “downloaded”  to  the  Design  Tradeoff  Tool, 
which  permits  the  team  to  both  qualitatively  and  quantitatively  evaluate  requirements  relative  to 
each  other  and  to  the  mission  of  the  aircraft.  The  resulting  information  will  allow  you  to  produce 
two  key  items.  First,  you  identify  a  set  of  system  “drivers”  for  your  cockpit,  which  cover  factors 
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such  as  capability,  range,  performance,  and  identify  the  necessary  cockpit  functions.  Second,  you 
then  use  this  information  to  develop  a  problem  statement  about  what  the  cockpit  should 
accomplish,  and  a  philosophy  of  how  your  cockpit  should  be  designed.  These  products  become 
the  basis  for  both  notional  and  actual  cockpit  designs. 

•  Build  Mission  Timeline 

Concurrently,  you  also  build  a  mission  timeline,  or  a  family  of  timelines,  to  determine  what  crew 
actions  within  the  cockpit  are  required  and  the  time  available  to  accomplish  the  mission.  This  is 
one  way  to  incorporate  pilot/crew  considerations  into  a  crew-centered  design  approach  as 
advocated  by  the  CCCD  Program.  Typically,  this  work  would  initially  be  performed  by  an 
Operations  Analysis  Group  using  computer  simulation  models  such  as  TAC  BRAWLER  or 
SUPPRESSOR.  These  digital  simulations  represent  the  approximate  mission,  threats,  and  aircraft 
factors  that  can  and  should  influence  your  cockpit  operation.  Usually,  these  simulations  do  not 
model  the  crew  actions  to  the  same  extent  that  they  represent  the  physical  systems.  Therefore,  the 
team  performs  an  analysis  using  a  Mission  Decomposition  Tool,  which  is  included  in  the 
CDS.  With  this  MDTOOL,  you  can  examine  the  flow  of  the  mission  from  a  crew  perspective. 
This  will  allow  you  to  try  out  several  approaches  to  accomplish  the  mission,  and  then  examine  how 
your  derived  requirements  compare  with  mission  drivers. 

•  Determine  System  Drivers 

•  Prepare  Notional  Baseline  Cockpit 

Armed  with  information  about  how  our  requirements  fit  the  mission,  you  can  now  work 
two  critical  aspects  of  cockpit  analysis: 

•  Determine  cockpit  design  drivers  in  terms  of  technology,  mission,  tactics,  human 
performance  attributes,  and  crew  requirements 

•  Prepare  a  notional  baseline  cockpit  design  (including  graphics  and  text) 

These  activities  will  be  accomplished  with  support  from  a  Cockpit  Product  Tool  which 
allows  you  to  collect  information,  while  creating  text  and  graphical  documents  about  your  cockpit 
design  approach.  This  material  can  be  used  within  your  company  and  throughout  the  SPO.  The 
specific,  tailored  product  applications  are  discussed  throughout  this  document. 

•  Provide  Input  to  Weapon  System  Specification 

Once  a  notional  cockpit  is  available,  you  can  begin  to  have  a  dramatic  influence  on  the 
aircraft  itself.  You  will  need  to  assemble  your  input  for  the  Weapon  System  Specification  (WSS). 
based  upon  our  notional  cockpit.  Again,  our  Cockpit  Product  Tool  will  allow  us  to  take  the 
derived  requirements  and  formulate  an  input  to  the  WSS  which  will  be  in  a  correct  and  usable 
format.  The  tool  simply  asks  you  to  review  and  clarify  each  requirement  (or  point  out  any 
deficiencies  from  associated  WSS  inputs)  before  consolidating  the  final  inputs  to  the  WSS.  By 
“staking  your  claim,”  other  system  and  subsystem  designers  now  know  that  the  cockpit  has  up¬ 
front  requirements,  which  must  be  represented  in  each  of  their  subsystems. 

•  Determine  Preliminary  Information  and  Control  Requirements 

•  Develop  Initial  Crew  System  Specification 

Another  critical  activity  is  that  of  the  initial  information  and  control  requirements  analysis. 
In  this  analysis,  you  examine  mission  and  crew  requirements  to  learn  what  the  crew  needs  to  look 
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at  and  control  within  the  cockpit,  for  each  specific  task  to  be  performed  in  the  design  mission.  Our 
Information  and  Control  Requirejccnts  Analysis  Tool  assists  you  in  creating  these 
requirements.  The  outcome  of  this  analy.w  then  flows  into  the  Cockpit  Product  Tool,  to  allow 
the  team  to  prepare  an  initial  Cockpit  Svstun  Specification  (CSS).  In  this  specification,  you  begin 
to  detail  the  elements  of  our  cockpit  from  a  requirements  standpoint,  based  upon  an  initial  Action/ 
Information  Requirements  Analysis.  Again,  this  “claim”  deepens  the  understanding  of  your  crew 
system  requirements  by  all  the  air  vehicle  system  and  subsystem  designers. 

•  Determine  Cockpit  Layout 

While  the  above  analyses  provide  requirements  for  the  rest  of  the  system,  they  also  help  to 
formulate  the  cockpit  layout.  You  prepare  the  layout  using  CDS’s  Crew  Station  Geometry 
Tool,  which  transforms  analysis  results  and  the  notional  cockpit  into  a  representative, 
dimensionalized  physical  cockpit  layout,  which  begins  to  approach  the  desired  individual  control 
and  display  design  elements  of  the  cockpit.  Our  database  of  past  cockpit  layouts  assists  you  in 
making  recommendations  for  the  cockpit’s  geometry,  layout,  and  dimensions. 

•  Evaluate  Results  of  Up-Front  Analysis 

The  results  of  these  analyses  and  preliminary  designs  will  need  to  be  re-examined  through 
the  Design  Tradeoff  Tool  to  ensure  that  initial  tradeoffs  remain  aligned  with  any  evolving 
requirements.  The  outcome  of  this  tradeoff  analysis  allows  you  to  change  the  crew  system 
requirements  or  the  physical  layout  of  the  cockpit.  In  turn,  this  leads  to  an  update  of  the  Cockpit 
System  Specification  (CSSi  and  an  improved  set  of  requirements. 

•  Complete  Design  Requirements  Document 

Upon  conclusion  of  the  Up-Front  Analysis,  you  will  need  to  formally  produce  a  Design 
Requirements  Document  (DRDL  which  contains  your  philosophy  of  cockpit  development  and 
summarizes  the  results  of  the  analyses  and  tradeoff  studies  performed  to  date.  This  DRD  becomes 
the  basis  by  which  you  further  refine  the  cockpit  design  via  CCCD  analysis,  design,  and  evaluation 
activities. 


•  Publish  Initial  Traceability  Report 

Throughout  the  Up-Front  Analysis  activities,  you  should  trace  and  document  (at  least  once) 
the  desired  cockpit  requirements.  Currently,  DTM  electronically  handles  the  associated 
traceability  files,  which  your  engineers  have  updated  each  step  of  the  way,  recording  how  and  why 
each  decision  was  made.  These  files  can  be  reviewed  independently,  or  consolidated  with  tradeoff 
decisions  and  cockpit  descriptions,  to  produce  an  initial  T raceabilitv  Document. 


GENERAL 

While  the  Up-Front  Analysis  and  tradeoffs  take  place,  the  team  gains  an  in-depth 
assessment  of  the  requirements  for  both  crew  and  mission.  With  this  new  insight,  you  will  also  be 
able  to  compose  a  Program  Plan  that  implements  the  remainder  of  the  cockpit  development 
program.  You  accomplish  this  with  support  from  a  Program  Planning/Scheduling  Tool 
(PPST).  This  tool  contains  a  database  of  planning  and  resource  information  for  access  by 
activities  throughout  the  program. 
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PROGRAM  PLANNING 


To  plan  an  effective  cockpit  development  program,  you  need  to  know  a  great  deal  about  the 
specific  crew  and  mission  requirements,  be  able  to  accurately  estimate  the  design  team’s  level  and 
balance  of  manpower  and  experience,  and  be  able  to  prepare  a  realistic  budget.  Having  already 
accomplished  Up-Front  Analysis  and  tradeoff  functions,  you  need  to  develop  a  sense  of  what  is 
needed  for  the  remainder  of  the  program.  Again,  background  information  contained  in  CDS 
Databases  can  provide  examples  of  scheduling,  requirements,  activities,  and  similar  cockpit  system 
developments.  These  databases  assist  you  in  planning,  but  the  experience  within  your  team  will 
likely  dictate  who  you  assign  to  lead  and  perform  the  balance  of  activities  within  the  program.  The 
Program  Planning/Scheduling  Tool  will  guide  your  development  of  plans  and  schedules. 

Two  main  functions  of  the  Program  Planning/Scheduling  Tool  are  the  selection  and 
scheduling  of  activities.  Both  need  to  be  performed  concurrently,  to  allow  for  a  coherent  plan 
which  meets  program  cost,  schedule,  and  quality  objectives. 

•  Develop  Program  Plan 

Within  the  program  plan,  the  order  and  extent  of  CCCD  Process  activities  depend  upon 
how  well  the  crew  and  mission  requirements  are  developed.  The  planning  function  helps  you  to 
reflect  the  current  requirements  and  cost  during  the  major  technical  part  (i.e.,  second  phase)  of  the 
CCCD  Process.  This  function  also  delves  into  other  support  areas,  such  as  Integrated  Logistics 
Support  (ILS)  activities,  helping  you  to  decide  how  to  best  allocate  the  available  time.  Again, 
decisions  of  this  type  are  only  as  good  as  the  experience  of  the  team  and  your  understanding  of 
requirements.  In  conjunction  with  the  Program  Planning/Scheduling  Tool,  you  will  use 
various  other  tools  such  as  the  Design  Tradeoff  Tool  for  tradeoffs  and  the  Cockpit  Product 
Tool  to  produce  documents  which  define  and  elaborate  program  plans,  decisions,  and  schedules. 

One,  two,  or  possibly  three  documents  critical  to  cockpit  development  could  be  created 
using  the  Program  Planning/Scheduling  Tool.  The  primary  document  prepared  under  the 
Program  Plan  is  commonly  known  as  the  Human  Engineering  Program  Plan  recommended  by 
MIL-H-46855.  This  document  elaborates  how  the  analysis,  design,  and  evaluation  activities 
performed  by  your  personnel  integrate  into  the  design  of  the  entire  weapon  system.  The  CDS 
supports  this  planning  function,  which  allows  you  to  select  tradeoff  analysis  requirements  from  a 
list  of  activities,  and  helps  you  report  the  critical  path  for  your  project.  This  type  of  information 
enables  you  to  schedule  your  team  for  parallel,  interdependent  activities. 

•  Develop  Dynamic  Simulation  Plan 

•  Complete  Dynamic  Simulation  Plan 

The  second  document  will  only  be  developed  if  the  program  plan  requires  pilot-in-the-loop 
simulation.  If  so,  then  a  Dynamic  Simulation  Plan,  more  commonly  called  out  as  the  Human 
Engineering  Dynamic  Simulation  Plan  in  MIL-H-46855,  will  be  created.  That  document 
establishes  the  methodology,  approach,  data  analysis,  and  scheduling  of  various  types  of  piloted 
simulation.  The  CDS  provides  examples  showing  the  type  and  number  of  simulations  needed, 
based  on  crew/mission  requirements  and  budget. 

•  Develop  Flight  Test  Plan 

•  Complete  Flight  Test  Plan 

The  third  document  that  may  be  needed  is  the  Flight  Test  Plan,  more  commonly  called  out 
as  the  Human  Engineering  Test  Plan  in  MIL-H-46855.  Again,  if  the  plan  calls  for  flight  testing. 
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then  CDS  provides  assistance  in  the  development  of  that  document.  Usually,  this  document  is  not 
produced  until  the  design  is  formalized  somewhat  later  in  the  program.  Another  component  of  the 
overall  CCCD  Program,  not  currently  resident  as  part  of  the  CDS,  is  a  support  tool  for  cockpit 
evaluation  during  flight  test,  including  guidance  for  preparing  the  cockpit  flight  test  plan.  That  tool 
is  called  the  Performance  Assessment  and  Workload  Evaluation  System  (PA WES).  It 
was  purposely  segregated  from  the  CDS  because,  while  cockpit  development  is  largely  an  industry 
activity,  the  test  role  is  shared,  with  the  Government  taking  on  a  significant  role  in  planning  and 
performing  flight  test.  Accordingly,  stand-alone  support  for  cockpit  flight  test  is  being  developed, 
although  integrating  it  into  the  CDS  is  feasible. 

•  Complete  Program  Plan 

Near  the  time  when  the  Up-Front  Analysis  activities  are  completed,  you  will  complete  the 
Program  Plan.  Your  team,  advised  by  the  program  plan  information,  now  begins  its  respective 
technical  work  (which  we  categorize  into  “bins”  of  analysis,  design,  and  evaluation  process  activi¬ 
ties).  As  you  can  see  from  the  Map,  each  of  these  functional  areas  begins  in  earnest  at  this  point. 
Another  facet  of  CDS  is  that  it  provides  you,  through  either  the  DTM  or  the  Program 
Planning/Scheduling  Tool,  a  way  to  track  the  activities,  or  to  read  an  engineer’s  status  report 
electronically  so  that  you  can  effectively  manage  your  program.  We  do  not  imply,  however,  that 
you  do  this  without  direct  contact  with  your  team.  The  CDS  merely  makes  that  tedious  job  of 
tracking  somewhat  easier  to  manage.  Since  there  are  always  tight  time  constraints,  effective 
oversight  of  your  progress  relative  to  plan  is  needed. 


CREW  SYSTEM  ANALYSIS 

The  following  detailed  map  (Figure  A-6)  of  Crew  System  Analysis  will  help  guide  you 
through  this  section  of  the  document. 

Upon  receiving  the  information  from  both  the  Design  Requirements  Document  and  the 
Program  Plan,  with  the  specifics  of  how  and  when  each  analysis  activity  takes  place,  and  the 
earlier  Up-Front  Mission  Analysis,  which  provides  a  starting  point,  your  team  is  ready  to  start  two 
critical  activities: 

•  Perform  Mission  Profile  Analysis 

•  Complete  Mission  Profile  Analysis  Report 

First,  you  would  access  the  information  from  the  earlier  mission  analysis  information  to 
provide  the  basic  elements  of  the  creation  of  a  Mission  Profile.  Armed  with  this  information 
members  of  the  design  team  (particularly  with  operational  experience  of  the  design  mission)  would 
“brainstorm”  a  set  of  appropriate  mission  events  for  placement  in  the  mission  through  a  technique 
known  as  Concept  Mapping.  This  technique  has  shown  the  unique  ability  to  correlate  the  thoughts 
of  operational  experts  toward  consistent  expressions  of  subject  matter.  The  results  of  this  effort 
can  be  obtained  through  the  usage  of  the  Concept  Mapping  Tool.  Once  the  correlated  mission 
events  are  obtained  they  can  be  placed  into  the  MDTOOL  as  a  means  to  implement  a  top-level 
mission  profile  for  analysis.  You  can  manipulate  the  variables  to  conform  to  the  actual  timing, 
environmental,  and  threat  characteristics  of  the  intended  mission  set.  This  activity  concludes  with  a 
time-tagged,  graphic  portrayal  of  all  mission  events  which  are  assumed  to  be  critical  to  cockpit- 
related  events.  This  information  (on  paper  and  in  electronic  media)  can  be  used  for  IPD  personnel 
for  planning  the  subsequent  piloted  simulations  for  a  “higher  level”  of  verification  and  an  early 
evaluation  of  realism  and  adequacy. 

•  Perform  Life  Support  Analysis 
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•  Perform  Escape  Analysis 

During  the  project,  an  analytical  look  at  the  intricacies  of  the  Life  Support  System  will  be 
made  to  reveal  the  important  integration  factors  determining  outcome  of  cockpit  design.  We  only 
acknowledge  a  great  need  for  this  analysis  and  do  not  delve  into  any  further  details  here.  Escape 
System  and  Life  Support  technologies  are  currently  being  advanced  in  separate  advanced 
development  projects  within  the  DoD.  The  elements  of  crew  protection  (e.g.,  life  support 
equipment,  escape  systems,  environmental  control)  are  critical  to  all  cockpit  development  projects, 
especially  for  combat  systems  that  employ  ejection  seats  or  escape  capsules,  and  so  the  CCCD 
Process  provides  for  their  consideration  during  Crew  System  Analysis. 

•  Perform  Reach  Analysis 

During  the  initial  look  into  the  cockpit  layout  and  integration,  it  is  vitally  important  that  a 
detailed  analysis  of  crew  reach  requirements  is  performed,  to  assure  full  access  to  controls  and 
displays  for  the  anticipated  size  range  of  the  aircrew  population.  This  is  performed  using  a  reach 
analysis  computer  model  called  COMBIMAN,  which  is  supported  by  CDS  and  is  already  in  use 
in  the  aircraft  industry  as  a  stand-alone  cockpit  accommodation  tool.  A  clear  determination  of  these 
requirements  can  be  accessed  versus  the  notional  cockpit  layout,  and  changes  can  be  implemented 
interactively  to  understand  their  effects.  Results  from  this  analysis  are  used  for  cockpit  re-design 
and  serve  as  critical  inputs  for  mock-up  analysis  activities.  As  a  further  verification  of  reach  and 
clearances  (usable  also  for  the  Escape  Analysis),  you  also  have  on-line  access  to  the  Cockpit 
Accommodation  Database,  which  can  quickly  check  on  reach  and  vision  clearances,  based  on 
physical  measurements  of  subjects  seated  in  specific,  existing  military  aircraft  cockpits. 

•  Perform  Mission  Scenario  Analysis 

•  Complete  Mission  Scenario  Analysis  Report 

You  then  prepare  mission  scenarios  for  each  of  the  mission  profiles  created.  This  activity 
requires  you  to  provide  a  detailed  script  about  mission  activities  for  each  crew  member  and  to 
include  system  and  environmental  interaction.  Specific  mission  scenarios  will  be  elaborated  in 
detail  for  all  the  mission  segments  that  require  analysis.  Several  databases  will  be  at  your  disposal 
to  help  formulate  the  proper  scripts  within  the  MDTOOL.  These  scripts  include  normal, 
unexpected,  and  emergency  conditions  that  put  the  crew  system  through  as  many  (analytically 
stimulating)  situations  as  possible.  These  same  conditions  will  be  used  during  Crew  System 
Evaluation,  and  the  scripts  can  be  sent  to  the  evaluation  team  via  MDTOOL’ s  electronic  output 
files  or  in  a  paper  format. 

•  Perform  Functional  Flow  Analysis 

•  Complete  Functional  Flow  Analysis  Report 

Now  that  you  have  defined  the  mission  purpose,  profiles,  and  top-level  activities  that  must 
take  place,  it  is  time  to  begin  breaking  down  all  facets  of  the  mission  using  Functional  Flow 
Analysis,  which  simply  breaks  down  tasks  to  their  lowest  level.  This  is  easily  accomplished  by 
using  a  graphical  block  diagram  approach  from  within  the  Functional  Flow  Tool.  This  allows 
you  to  functionally  break  out  the  mission  into  its  major  segments  (block  I),  all  functional  activities 
within  each  segment  (block  II),  and  each  distinct  task  within  each  function  (block  III).  Note  that 
block  IE  tasks  can  be  delineated  to  as  many  levels  as  required.  This  breakout,  along  with  a  written 
description  at  each  level,  will  prepare  you  for  the  next  activity  of  Action/Information  Requirements 
Analysis  described  below.  The  lowest  level  tasks  will  become  the  basis  for  the  database  of  tasks 
that  will  be  used  within  the  Timeline  Management  Tool  throughout  all  following  analysis 
activities.  When  this  activity  has  been  completed,  you  will  be  able  to  discern  new  crew  system 
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requirements  and  update  both  the  Cockpit  System  Specification  (through  the  Cockpit  Product 
Tool)  and  the  Cockpit  Traceability  Report  (through  the  DTM  and  the  Cockpit  Product  Tool), 
both  of  which  provide  the  basis  for  a  program  review. 

•  Perform  Action  Information  Requirements  Analysis 

•  Complete  Action  Information  Requirements  Analysis  Report 

The  development  and  analysis  of  Action/Information  Requirements  is  the  most  critical 
activity  for  the  development  of  control  and  display  interfaces.  These  Action/Information 
Requirements  are  “cornerstone”  elements  from  which  you  design  the  controls  and  displays.  Our 
Information  and  Control  Requirements  Analysis  Tool  helps  you  to  access  databases  that 
contain  requirements  for  the  type  and  class  of  aircraft  in  your  development  or  modification  project. 
The  database  (from  within  the  Timeline  Management  Tool)  started  during  Functional  Flow 
Analysis  continues  to  store  information  that  will  be  added  to  here.  You  provide  Action/Information 
Requirements  for  each  (sequential  and  parallel)  mission  task,  to  associate  and  depict  specific 
requirements  with  each  phase  of  the  design  mission.  You  can  look  at  the  results  from  several 
perspectives  (i.e.,  by  crew  member,  by  sub-system,  or  by  individual  requirement)  to  gain  an 
understanding  of  how  the  requirements  become  distributed  to  crew  members  and  subsystems. 
Guidance  on  this  topic  is  also  derived  from  the  System  Operational  Requirements  Document  and 
the  Cockpit  Design  Philosophy  that  will  have  already  been  established.  This  information  will  help 
you  to  develop  the  cockpit  controls  and  displays  and  also  to  write  an  enhanced  Cockpit  System 
Specification  with  a  technical  justification  of  the  design  rationale  documented  in  an  updated  Cockpit 
Traceability  Report. 

•  Develop  Task  Descriptions 

•  Complete  Task  Descriptions  Report 

Armed  with  a  deeper  understanding  of  the  Action/Information  Requirements  for  each  task 
throughout  the  mission,  you  can  now  further  characterize  the  aircrew  tasks  with  information  from 
the  same  database  used  in  each  of  the  last  two  analysis  activities.  You  lay  out  the  sub-activities 
within  each  task,  and  then  estimate  the  gross  timing  parameters.  This  activity  allows  you  to 
reassess  each  of  the  lowest-level  tasks  in  light  of  fully  defined  Action/Information  Requirements. 
You  also  enter  written  descriptions  that  go  along  with  the  abbreviated  tasks  into  a  CDS  Database 
within  the  Timeline  Management  Tool.  This  information  will  be  used  to  further  the  design 
and  evaluation  of  controls  and  displays  and  establishes  the  basis  for  future  analysis. 

•  Perform  Function  Allocation  Trade  Analysis 

•  Complete  Function  Allocation  Trade  Analysis  Report 

The  team  has  now  developed  the  aircrew  tasking  in  enough  detail  to  make  decisions 
regarding  the  implementation  of  the  required  system  functions  between  crew  members  and 
subsystems,  using  Function  Allocation  Trade  Analysis  supported  by  the  Design  Tradeoff  Tool. 
A  CDS  database  allows  you  to  use  its  stored  information  to  create  trades  or  to  reassess  trades  using 
the  results  from  this  analysis.  The  CDS  also  is  used  to  retrieve  trade  study  information  from 
similar  aircraft,  and  you  can  further  review  the  initial  allocation  using  one  of  several  variants  of  the 
Fitt’s  List  (a  list  of  attributes  associated  with  strengths  of  man  versus  machine).  This  analysis 
concludes  with  a  recommended  crew  member/subsystem  allocation  of  work,  with  an  updated 
database,  and  more  importantly  with  a  new  description  of  the  key  requirements  for  the  various  air 
vehicle  subsystem  designers  so  that  they  can  better  factor  the  analyzed  crew  system  requirements 
into  their  designs.  Some  updated  requirements  may  dictate  that  new  algorithms  be  created  to 
support  a  mission  or  aircrew  activity,  which  earlier  may  have  been  performed  by  crew  members  or 
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by  dedicated  subsystems  (with  or  without  monitoring  or  action  by  the  pilot/crew).  In  either  case, 
the  design  team  needs  this  information  to  assess  controls  and  displays.  The  revised  database  (from 
within  the  Timeline  Management  Tool)  also  supports  Task  Timeline  Analysis,  which  follows. 

•  Perform  Task  Timeline  Analysis 

•  Complete  Task  Timeline  Analysis  Report 

Task  Timeline  Analysis  is  performed  with  support  from  the  CDS’s  Timeline 
Management  Tool.  The  Timeline  Management  Tool  enables  the  team  to  assess  the  new 
baseline  cockpit  design  (which  will  have  been  passed  to  you  through  the  Cockpit  Product 
Tool),  to  ensure  that  the  new  design  meets  the  updated  requirements  at  the  pilot/crew  task  level. 
Here,  you  define  specific  requirements  such  as  discrete  motion,  vision,  speech,  etc.  Also,  you 
will  update,  as  needed,  the  physical  aspects  for  each  task  such  as  reach  distances  (which  you  now 
have  via  the  cockpit  mockup  evaluation,  and  supplemented  by  the  COMBIMAN  analysis  of  the 
cockpit  geometry),  types  of  reach,  visual  attributes,  forces,  releases,  rotation  angles,  etc.  You 
need  to  determine  accurate  and  appropriate  time  values  for  each  crew  member  task,  based  upon 
known  subsystem  designs,  similar  subsystem  designs,  or  Methods-Time  Measurement  (MTM) 
estimates  (contained  within  the  Timeline  Management  Tool).  You  are  then  in  a  position  to 
identify  the  total  system  delay  times  and  sequential  mission  timing,  derived  and  analyzed  from 
within  the  Timeline  Management  Tool,  to  characterize  the  individual  crew  actions.  This  data 
will  be  used  to  formulate  piloted  simulator  testing  scenarios,  to  evaluate  the  cockpit  design 
attributes  in  light  of  requirements,  and  to  update  the  Action/Information  Requirements  Analysis  and 
accomplish  the  Task  Workload  Analysis. 

•  Perform  Update  of  Action/Information  Requirements  Analysis 

•  Complete  Update  of  Action/Information  Requirements  Analysis  Report 

With  the  newly  developed  CDS  Timeline  Database  (from  within  the  Timeline 
Management  Tool),  which  includes  detailed  timing  aspects  necessary  to  meet  the  revised 
mission  and  cockpit  requirements,  you  perform  a  complete  update  of  the  Action/Information 
Requirements  to  discover  if  any  new  requirements  exist,  or  if  any  old  requirements  can  be 
efficiently  combined  (through  display/control  interfaces).  The  results  should  indicate  that  the 
cockpit  design  (concurrent  development  via  the  Crew  System  Design  and  Crew  System  Evaluation 
activities)  continues  to  satisfy  the  established  crew  requirements.  If  not,  then  the  design  will  need 
to  be  changed  and  the  rationale  for  change  will  be  recorded  in  the  DTM.  You  provide  the  results, 
and  your  recommendation  for  change,  to  the  design  team.  They  will  trade  your  new  requirement 
against  all  related  subsystem  designs  and  reach  a  formal  cockpit  design  compromise  decision 
through  the  use  of  a  Design  Tradeoff  Tool  discussed  in  a  later  section. 

•  Perform  Task  Workload  Analysis 

•  Complete  Task  Workload  Analysis  Report 

While  updating  the  Action/Information  Requirements,  the  team  needs  to  perform  a  detailed 
Task  Workload  Analysis.  Using  stored  information  from  the  Timeline  Management  Tool, 
you  will  annotate  each  task  with  the  estimated  discrete  task  workload,  continuous  load,  and  mode 
attributes.  Then,  you  assign  each  task  an  appropriate  workload  attribute  (mission  segment 
dependent),  and  run  a  time-required-versus-time-available  digital  workload  model.  The  CDS’s 
Workload  Analysis  Tool  will  assist  in  performing  this  activity.  Data  flows  from  Timeline 
Management  Tool  directly  into  the  Workload  Tool  to  allow  you  to  enter  the  proper  variables 
from  within  the  model.  Example  CDS  Databases  will  assist  you  in  performing  this  analysis. 
The  results  of  this  activity  will  be  passed  to  your  design  team  who  will  look  for  trends  by  crew 
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member.  Later,  you  will  compare  these  results  to  simulator  values,  to  verify  the  adequacy  of  your 
design  and  your  analytic  predictions  (if  necessary). 


GENERAL 

The  results  from  these  CDS  Analysis  Tools  will  have  a  significant  impact  upon  both  the 
Crew  System  Design  and  the  Crew  System  Evaluation  activities  within  CCCD  Process.  Our  intent 
is  to  provide  both  the  methodology  and  the  tools  with  which  you  derive  and  verify  requirements. 
When  done  up  front  and  quickly,  your  analysis  can  have  a  profound  effect  on  requirements  and 
cockpit  design.  Early  identification  and  analysis  of  the  aircrew’s  interactions  with  the  cockpit,  as 
they  both  evolve,  becomes  a  tangible  basis  on  which  the  various  Integrated  Product  Teams  can 
interact  with  your  cockpit  team,  and  assure  that  a  crew-centered  design  approach  will  satisfy  the 
cockpit  user  needs. 


CREW  SYSTEM  DESIGN 

Typically,  you  will  prepare  a  functional,  although  notional,  cockpit  design.  This  is 
normally  followed  by  a  more  detailed  reflection  of  your  preliminary  design  during  the  first  month 
of  the  program.  However,  the  real  inception  of  detailed  design  must  come  from  the  results  of 
analytical  activities  described  earlier.  This  section  will  guide  your  team  through  the  use  of  those 
results  and  how  to  further  the  design  into  specification-ready  material.  The  following  detailed  map 
(Figure  A-7)  of  Crew  System  Design  will  show  you  how  the  activities  in  this  section  of  the 
document  take  place. 

•  Assess  Results  of  Functional  Flow  Analysis 

•  Assess  Results  of  Action/Information  Requirements  Analysis 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

While  the  origins  of  crew  system  design  begin  with  Up-Front  Analysis  and  a  notional 
cockpit  baseline,  iterative  design  begins  with  the  results  from  Functional  Flow  Analysis  using  the 
Timeline  Management  Tool,  the  product  of  which  is  housed  in  Cockpit  Product  Tool  for 
review.  A  critical  review  will  help  to  structure  the  makeup  of  the  cockpit  design  by  allowing  the 
design  team  to  understand  the  flow  of  lower-level  crew  tasks.  With  this  understanding,  the  team 
can  begin  to  design  a  cockpit  layout  (using  the  Crew  Station  Geometry  Tool)  and  recommend 
all  the  cockpit  information  and  control  requirements  using  the  Information  and  Control 
Requirements  Analysis  Tool,  both  of  which  are  necessary  for  interface  design  and  vital  to 
Action/Information  Requirements  Analysis.  With  the  results,  the  design  team  adjusts  the 
COMBIMAN  model  and  the  cockpit  mockup,  which  is  soon  slated  for  evaluation.  Using  the 
Cockpit  Product  Tool,  you  produce  an  updated  Cockpit  System  Specification  and  will  be  ready 
for  a  program  review.  Your  efforts  to  develop  this  new  Cockpit  System  Specification  will  need  to 
be  documented.  You  will  use  DTM  to  refine  the  cockpit  traceability  record  of  the  design  to  date. 

•  Prototype  Controls  and  Displays 

The  results  of  the  Action/Information  Requirements  Analysis  provide  the  next  opportunity 
for  design  change.  Information  retrieved  from  the  Cockpit  Product  Tool  and  Design 
Traceability  Manager’s  intermediate  traceability  files  will  shape  how  the  team  devises  various 
cockpit  display  formats  before  they  begin  to  look  at  specific  interface  controls  or  displays.  First, 
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.  Process  Map  -  Crew  System  Design  Activities  (2  of  2) 


you  inspect  requirements  by  functional  categories:  i.e.,  navigation,  weapon  fire  control,  target 
acquisition,  etc.  With  those  functional  requirements,  you  are  in  position  to  envision  displays  and 
controls  that  meet  the  requirements.  In  particular,  you  review  the  products  from  the  updated 
Action/Information  Requirements  Analysis,  checking  analytical  results  to  see  how  they  vary  from 
original  estimates.  The  team  will  cross-check  its  evolving  designs  against  mission  requirements 
(airspeed,  missile  launch  envelope,  map  symbology,  etc.)  at  representative  times  and  places. 
Lastly,  you  review  what  information  each  crew  member  uses  at  various  times  in  the  mission.  For 
instance,  what  information  does  the  pilot  need  to  operate  the  aircraft  during  low-level  penetration? 
Do  both  crew  members  review  threat  assessment  variables  prior  to  launching  missiles  beyond 
visual  range? 

Your  team  will  begin  building  several  iterations  of  a  design  for  each  area  of  interest.  The 
Control  and  Display  Development  Tool  portrays  each  interface  in  varying  style,  layout,  and 
color  (if  required).  The  CDS  provides  references  that  can  help  structure  and  standardize  the  design 
process  through  rules  about  font  size,  color,  interface  methods,  symbols,  etc.  Using  this  basic 
design  approach,  your  team  will  create  a  small  number  of  candidate  designs  with  the  Control  and 
Display  Development  Tool,  choosing  one  or  two  for  rapid  prototyping  evaluation. 

•  Perform/Complete  Rapid  Prototyping  Evaluation 

While  technically  an  evaluation  activity,  Rapid  Prototyping  Evaluation  is  also  often  thought 
of  as  a  design  activity,  as  the  two  uses  of  this  method  overlap.  This  evaluation  further  refines  the 
requirements,  which  will  be  documented,  and  spawns  new  designs.  This  is  the  key  to  interactive 
cockpit  design:  iterative  design  tradeoff  cycles. 

•  Evaluate  Results  of  Task  Description  Analysis,  Rapid  Prototyping  Evaluation,  Life 
Support  Analysis,  Escape  Analysis  and  Control/Display  Logic 

In  this  CCCD  Process  activity,  you  contrast  the  results  of  the  Rapid  Prototyping  Evaluation 
against  the  Task  Description  Analysis  activity.  This  will  provide  tradeoff  attributes  for  the  Design 
Tradeoff  Tool,  which  the  team  will  use  to  justify  how  the  updated  design  attributes  should 
satisfy  the  crew  and  mission  requirements.  In  addition,  design  information  (from  the  tradeoffs), 
provides  critical  inputs  to  a  Function  Allocation  Trades  Study. 

•  Update  Baseline  Cockpit  Design 

These  efforts  will  likely  lead  to  redesigning  the  cockpit.  This  particular  iteration  yields  in- 
depth  design  features,  such  as  interface  design  descriptions,  which  become  part  of  the  Cockpit 
Mechanization  Document,  a  “living”  design  guide  for  the  team.  With  each  redesign,  one’s 
understanding  of  the  weapon  system  and  its  interface  expands.  One  of  the  tools  that  will  assist  you 
in  the  development  of  the  interface  design  is  the  Mechanization  Logic  Tree  Tool.  This  tool 
helps  you  to  structure  the  design  by  showing  the  logical  interfaces  among  applicable  subsystems. 
The  logic  diagrams  will  save  time  and  increase  your  ability  to  influence  the  other  subsystem 
designs  by  determining  which  interfaces  need  to  become  directly  connected  with  the  crew  system 
and  what  supporting  elements,  such  as  algorithms,  need  to  be  specified. 

•  Build  Cockpit  Mechanization  Document 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

Again,  as  with  almost  any  design  activity,  the  results  of  this  effort  lead  to  a  Cockpit 
Mechanization  Document  and  an  updated  Cockpit  System  Specification.  Generally,  only  style  and 
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depth,  not  content,  differentiate  the  Cockpit  Mechanization  Document  from  the  Cockpit  System 
Specification.  Both  are  a  formal  part  of  the  CCCD  Process  and  are  supported  by  the  CDS.  The 
CDS  stores  the  new  Cockpit  Mechanization  Document  and  Cockpit  System  Specification  and 
stores  the  design  team  updates  in  the  Cockpit  Traceability  Report. 

•  Evaluate  Results  of  Function  Allocation  Trade  Analysis,  Mockup  Evaluation  Results,  and 
the  Cockpit  Mechanization  Document  to  Update  the  Cockpit  Design 

Once  you  have  reviewed  the  results  of  the  Function  Allocation  Trade  Study  and  the 
Mockup  Evaluation  Report,  you  would  use  the  Design  Tradeoff  Tool  to  ensure  that  the 
functional  allocation  between  individual  crew  member  and  system  meets  the  updated  requirements. 
This  may  cause  you  to  rethink  some  of  the  functional  design  characteristics.  Typically,  changing 
the  crew  member  of  a  system  or  sub-system  only  affects  the  control  interface.  Likewise,  whenever 
an  informational  aspect  changes,  so  will  the  crew  interface,  even  if  it  affects  nothing  more  than  an 
input  from  a  pilot’s  situational  awareness  perspective. 

•  Redesign  Cockpit 

•  Update  Cockpit  Mechanization  Document 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

Any  change  requires  a  potential  cockpit  redesign,  including  an  update  of  the  Cockpit 
Mechanization  Document,  Cockpit  System  Specification,  and  Cockpit  Traceability  Report.  This 
new  mechanization  feeds  Rapid  Prototyping  Evaluation  and  forms  the  basis  for  Task  Timeline 
Analysis. 

•  Evaluate  Results  of  Task  Timeline  Analysis  and  Rapid  Prototyping  Evaluation 

•  Redesign  Cockpit 

•  Update  Cockpit  Mechanization  Document 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

Next,  use  the  Design  Tradeoff  Tool  to  compare  and  contrast  results  from  Task  Timeline 
Analysis  and  Rapid  Prototyping  Evaluations  to  enhance  the  cockpit  design.  The  changes  will 
require  you  to  redesign  the  cockpit,  document  the  redesign  through  an  updated  Cockpit 
Mechanization  Document,  and  Cockpit  Traceability  Report.  You  are  now  positioned  to  present  and 
explain  the  resulting  Cockpit  Mechanization  Document  to  the  project’s  simulation  and  evaluation 
personnel  to  prepare  them  for  further  cockpit  evaluation  issues. 

•  Evaluate  Results  of  Task/Workload  Analysis,  updated  Action/Information  Requirements, 
and  the  updated  Cockpit  Design 

Next,  you  apply  the  results  of  the  Task/Workload  Analysis  and  Action/Information 
Requirements  Analysis  (as  well  as  current  design  characteristics)  to  critique  and  revise  the  design. 
You  also  will  update  the  Cockpit  Mechanization  Document.  This  begins  yet  another  iteration  of 
design  rapid  prototyping  and  evaluation.  v  hen  evaluate  new  against  old  workload  predictions 
to  see  if  the  pilot  and  other  crew  member  u  come  more  manageable. 
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•  Redesign  Cockpit 

•  Update  Cockpit  Mechanization  Document 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

Cockpit  redesign  follows,  as  do  changes  to  the  Cockpit  Mechanization  Document.  Cockpit 
System  Specification,  and  Cockpit  Traceability  Report  in  time  for  a  program  review.  Again,  you 
will-coordinate  the  Cockpit  Mechanization  Document  with  the  simulation  and  evaluation  personnel 
to  ensure  that  the  design  will  be  properly  evaluated. 

•  Evaluate  Results  from  Part-Task  Simulation  Evaluation  #1 

The  results  of  the  first  Part-Task  Simulation  will  be  prepared  in  two  ways,  as  a  Part-Task 
Simulation  Evaluation  Report,  and  as  an  intermediate  DTM  Traceability  File  that  explains  the 
reasons  behind  the  test  team’s  conclusions  and  recommendations.  You  then  reassess  the  crew 
system  design  in  light  of  those  conclusions  and  recommendations  as  well  as  inputs  from  the  overall 
design  team.  This  is  the  first  time  your  design  is  judged  against  real-time  crew  member 
expectations.  The  majority  of  the  cockpit  design  will  begin  to  change  in  terms  of  detailed  software 
interfaces.  This  can  be  both  in  visual  formats  and  symbology  to  the  crew  members  and  subsystem 
detailed  interfaces.  Potentially  these  changes  can  even  accommodate  evolving  requirements.  There 
will  be  many  duplications  of  informational  requirements  which  you  will  be  better  able  to  manage, 
with  support  from  the  CDS,  using  the  results  of  piloted  simulation.  This  will  also  be  an  excellent 
opportunity  to  apply  the  Design  Tradeoff  Tool. 

•  Perform/Complete  Rapid  Prototyping  Evaluation 

•  Evaluate  Results  From  Rapid  Prototyping  Evaluation 

•  Redesign  Cockpit 

•  Update  Cockpit  Mechanization  Document 

•  Update  Ccvl  pit  System  Specification 

•  Update  C  kr  Traceability  Repon 

The  previous  activities  employ  a  series  of  rapid  prototyping  design  iterations.  At  each  step, 
you  continue  to  evaluate  while  incorporating  changes  to  the  cockpit  design.  At  each  redesign,  you 
again  need  to  revise  the  Cockpit  Mechanization  Document.  Cockpit  System  SpccificalioiL  and 
Cockpit  Traceability  Report.  At  this  point,  you  review  the  updated  Cockpit  Mechanization 
Document,  and  resolve  issues  from  the  Part-Task  Simulation  Evaluation  Report  and  the  FplJb 
Mission  Simulation  Evaluation  Report,  with  the  simulation  and  evaluation  personnel,  again,  to 
ensure  that  the  design  is  progressing  well. 

•  Evaluate  Results  from  Part-Task  Simulation  Evaluation  #2 

•  Perform/Complete  Rapid  Prototyping  Evaluation 

•  Evaluate  Results  From  Rapid  Prototyping  Evaluation  and  Part-Task  Simulation 
Evaluation  #3 
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•  Perform/Complete  Rapid  Prototyping  Evaluation 

•  Evaluate  Results  From  Full-Mission  Simulation  Evaluation  #1 

At  this  point,  you  will  probably  know  the  results  from  the  second  Part-Task  Simulation 
Evaluation  Report,  in  which  case  you  need  to  reassess  its  conclusions  and  recommendations,  in 
light  of  how  you  designed  the  cockpit.  This  may  lead  to  another  rapid  prototyping  cycle  and  a 
revised  design  approach.  You  then  determine  what  changes  are  truly  necessary  for  the  cockpit 
design.  You  may  also  have  the  results  from  the  third  Part-Task  Simulation  Report  to  add  to  the 
Rapid  Prototyping  Evaluation  results.  At  this  point,  the  rapid  prototype  evaluation  team  should  re¬ 
evaluate  the  cockpit  prototype  controls  and  displays.  The  results  of  this  evaluation  should  coincide 
with  the  results  of  the  first  Full  Mission  Simulation  Evaluation  Report.  This  iterative  process  of 
evaluation  and  redesign  naturally  leads  to  a  more  robust  redesign,  with  the  obvious  human  factors 
problems  resolved  and  documented  along  the  way.  You  should  truly  have  your  first  full  look  at  all 
of  the  integration  factors  which  are  necessary  for  the  cockpit.  Until  this  point,  the  best  your  design 
could  hope  for  is  that  each  of  the  individual  subsystem  designs  had  been  through  enough  Rapid 
Prototyping  or  Part-Task  Simulation  Evaluation  cycles  to  “come  together.” 

•  Redesign  Cockpit 

•  Update  Cockpit  Mechanization  Document 

•  Update  Cockpit  System  Specification 

•  Update  Cockpit  Traceability  Report 

Once  the  cockpit  redesign  is  approved,  you  will  need  to  make  the  appropriate  changes  to 
the  Cockpit  Mechanization  Document,  Cockpit  System  Specification,  and  Cockpit  Traceability 
Report  before  the  scheduled  program  review.  At  that  point,  you  present  and  explain  the  critical 
Cockpit  Mechanization  Document  issues  in  advance  of  the  next  Part-Task  Simulation  Evaluation  (if 
any)  and  Full  Mission  Simulation  (if  any),  to  ensure  that  the  designs  will  be  evaluated  properly  in 
each  case. 


GENERAL 

The  above  iteration  and  interaction  cycle  can  be  performed  as  many  times  as  necessary 
(given  the  available  time  and  resources)  to  accomplish  design  goals.  Once  fully  developed  and 
proven,  the  CDS  will  enable  more  design  iterations  and  design  quality  checks.  We  chose  a 
representative  number  of  activities  to  show  that  sometimes  the  same  activities  take  on  a  slightly 
different  “twist,”  due  to  the  state  of  development  and  the  type  of  available  information  and  results 
from  analyses  and  tests.  Note  that  a  great  deal  of  activity  is  implied  for  both  the  analysis  and  the 
evaluation  activities. 


CREW  SYSTEM  EVALUATION 

Given  the  Design  Requirements  Document  direction  for  the  cockpit  and  a  Program  Plan 
with  the  specifics  about  each  evaluation  activity  to  take  place,  and  also  given  the  CCCD  analysis 
information  about  the  specific  evaluation  requirements,  you  will  be  ready  to  plan  and  perform  an 
orderly  progression  of  Crew  System  Evaluations.  As  in  current  practice,  evaluation  is  a  continuing 
activity  as  the  cockpit  design  evolves.  With  the  CCCD  Process,  the  evaluations  progress  in  an 
orderly  manner,  from  early  studies  that  would  be  totally  supported  by  the  CDS,  to  separate 
physical  and  lighting  mockups,  to  larger-scale  incorporation  into  avionics  hotbenches  and  flight 
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control  fixtures,  and  eventually  within  weapon  system  simulators  (including  ground-based  domed 
simulators  and  in-flight  simulators).  This  document  focuses  on  the  use  of  real-time  piloted 
simulation  for  verifying  the  crew  system  operability,  although  all  types  of  evaluation  will  be 
performed. 

The  following  detailed  map  (Figure  A-8)  of  Crew  System  Evaluation  will  help  guide  you 
through  this  section  of  the  document. 

•  Develop  Dynamic  Simulation  Plan 

•  Complete  Dynamic  Simulation  Plan 

If  the  Program  Plan  requires  pilot-in-the-loop  simulation  testing,  then  you  would  prepare  a 
Dynamic  Simulation  Plan  using  the  Program  Planning/Scheduling  Tool  and  Cockpit 
Product  Tool.  The  Dynamic  Simulation  Plan  will  document  the  methodology,  approach,  data 
analysis,  and  scheduling  of  cockpit  simulators.  Simulators  could  include  part-task  and  full- 
mission  simulation  devices.  As  previously  stated,  the  CDS  provides  some  assistance  in  planning 
the  number  and  nature  of  simulation  tests,  using  your  crew/mission  requirements  and  budget  as 
constraints. 

•  Plan  Part-Task  Simulation  Evaluations 

•  Plan  Full-Mission  Simulation  Evaluations 

Once  you  have  completed  the  Dynamic  Simulation  Plan,  you  begin  the  actual  preparation 
for  specific  Part-Task  Simulator  design  evaluations.  This  will  require  that  you  have  an  agreed- 
upon  baseline  of  the  design  to  evaluate.  Forma]  test  planning  can  be  developed  through  the 
Simulation  Test  Planning  Tool  where  a  database  of  ideas,  methodology,  tradeoff  of 
requirements,  and  data  analysis  techniques  can  be  accessed  to  promote  viable  evaluations. 
Similarly,  you  will  plan  for  any  Full-Mission  Simulation  evaluations  that  need  to  occur.  Actual 
evaluation  plans  will  be  produced  using  the  Cockpit  Product  Tool  to  conform  to  project- 
specific  formats. 

The  CCCD  Program  has  published  a  document  which  will  be  the  basis  for  this  tool 
development.  This  report  can  be  retrieved  from  DTIC  as  HSD-TR-90-007,  “Handbook  For 
Conducting  Pilot-In-The-Loop  Simulation  For  Crew  Station  Evaluation,”  Accession  number 
B141991. 

•  Plan  Mockup  Evaluation 

•  Plan  Rapid  Prototyping  Evaluations 

Other  evaluations  also  need  to  be  planned.  These  can  vary  in  nature.  For  example,  you 
will  plan  a  cockpit  mockup  evaluation  as  soon  as  the  team  formalizes  the  cockpit’s  preliminary 
design,  as  a.  further  check  against  your  COMBIMAN  analysis.  You  will  plan  the  Rapid 
Prototyping  Evaluations  that  were  previously  discussed  under  Crew  System  Design.  You  will  also 
prepare  the  overall  evaluation  team  to  stay  abreast  of  the  evolving  cockpit  analysis  results  and  the 
evolving  design  requirements.  With  support  from  the  CDS,  you  will  be  in  a  position  to  help  them 
to  develop  quick  methods  to  extract  evaluation  data  from  each  Rapid  Prototyping  Evaluation, 
especially  data  that  is  pertinent  to  the  needs  of  the  other  air  vehicle  subsystems. 

•  Perform  Part-Task  Simulation  Evaluations 

•  Perform  Full-Mission  Simulation  Evaluations 


69 


Figure  A-8.  Process  Map  -  Crew  System  Evaluation  Activities 


Throughout  the  life  of  the  program,  a  need  to  perform  (potentially  several)  Part-Task  and 
Full-Mission  Simulation  Evaluations  will  be  emphasized  to  place  your  design  under  realistic 
conditions  for  overall  system  integration.  With  the  use  of  Structured  Test  Plans  and  Procedures, 
accessible  on  the  CDS  via  the  Simulation  Test  Planning  Tool,  your  team  can  produce 
meaningful  results  in  time  for  re-use  by  other  subsystem  design  teams.  Typically,  the  simulation 
personnel  follow  the  direction  of  the  crew  system  Integrated  Product  Team  in  completing 
simulation  evaluations.  It  is  imperative  that  someone  from  the  your  design  team  be  responsible  for 
those  activities  so  the  data  can  be  analyzed  with  crew-centered  cockpit  design  as  the  driving  force. 

•  Complete  Part-Task  Simulation  Evaluation  Results 

•  Complete  Full-Mission  Simulation  Evaluation  Results 

All  evaluation  results,  whether  in  the  form  of  specific  conclusions  or  general  recommenda¬ 
tions,  will  be  inserted  into  the  design  evaluation  tradeoff  matrix  for  your  program.  Therefore,  you 
are  advised  to  use  a  mix  of  qualitative  and  quantitative  methods  to  strengthen  conclusions  and 
recommendations.  Further,  you  should  strive  to  include  timely  indications  and  trend  information 
to  show  how  the  crew  performs  the  mission,  rather  than  including  “data  dumps.”  Cost  and 
schedule  will  severely  constrain  your  ability  to  exhaustively  examine  all  of  the  simulator  test 
results.  The  CDS  provides  support  through  the  Simulation  Test  Planning  Tool,  which  helps 
you  to  create  a  balanced  series  of  appropriate  evaluations  that  conform  to  program  needs. 

•  Develop  Flight  Test  Plan 

•  Complete  Flight  Test  Plan 

If  the  Program  Plan  mandates  flight  testing,  you  will  prepare  a  Flight  Test  Plan  using  the 
CDS  Program  Planning  Tool  (or  PAWES)  and  the  CDS  Cockpit  Product  Tool.  This 
plan  documents  the  methodology,  approach,  data  analysis,  and  scheduling  of  various  types  of 
flight  test  activity  to  coincide  with  the  rest  of  the  flight  test  program. 

•  Plan  Developmental  Test  and  Evaluation  Flight  Tests 

•  Plan  Operational  Test  and  Evaluation  Flight  Tests 

Flight  test  evaluation  is  handled  differently  than  simulator  evaluation.  The  goal  of  flight 
testing,  both  Developmental  Test  and  Evaluation  (DT&E)  and  Operational  Test  and  Evaluation 
(OT&E),  is  to  measure  the  ability  of  the  system  or  subsystem  item  to  meet  the  original  require¬ 
ments.  DT&E  addresses  design  requirements,  whereas  OT&E  evaluates  against  operational  re¬ 
quirements.  While  the  two  evaluation  types,  DT&E  and  OT&E,  are  substantially  different  in 
nature  (and  include  increasing  participation  by  the  military  customer  in  planning  and  performing 
these  tests),  they  do  abide  by  similar  techniques  of  structured,  “by-the-book”  testing.  Where 
possible,  the  variables  are  controlled  and  the  test  items  are  isolated  in  an  attempt  to  validate  the  test 
article. 


In  the  future,  the  CDS  will  support  your  ability  to  plan  and  execute  flight  test  activities, 
based  on  crew  and  mission  requirements,  as  well  as  budget.  As  was  mentioned  above,  the  CCCD 
Program  Office  is  developing  a  cockpit  flight  test  support  tool  (PAWES)  for  that  purpose. 
PAWES  has  already  achieved  considerable  acceptance  from  the  flight  test  community  and  will  be 
undergoing  field  trials  at  the  Air  Force  Flight  Test  Center  during  FY94.  Hopefully,  a  similar  state 
of  user  acceptance  and  support  for  the  CDS  will  develop  within  the  industry  crew  system  design 
community  as  the  system  matures  and  is  proven  through  use  in  actual  applications. 
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SECTION  4.0:  CONCLUSION 

The  CCCD  Process  heavily  relies  upon  user  interaction  and  design  iteration  to  improve  the 
cockpit  design  product  quality.  Conversely,  we  believe  that  you  will  also  heavily  rely  upon  the 
CDS  and  its  supporting  toolset,  to  improve  your  cockpit  design  process.  With  your  help,  and  with 
your  constructive  recommendations  for  improvement,  we  will  be  able  to  make  a  joint  contribution 
to  advancing  the  overall  practice  of  crew  system  design. 
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ACRONYMS 


CAD 

Computer-Aided  Design 

CADAM 

Computer-Aided  Design  and  Manufacturing 

CALS 

Computer-Aided  Acquisition  and  Logistics  System 

CAM 

Computer-Aided  Manufacturing 

CAT 

Cockpit  Automation  Technology 

CATIA 

Trade  name  for  the  Dassault  CAD/CAM  package 

CCCD 

Crew-Centered  Cockpit  Design 

CD 

Compact  Disc 

CDDT 

Control  and  Display  Development  Tool 

CDRL 

Contract  Data  Requirements  List 

CDS 

Cockpit  Design  System 

CMT 

Concept  Mapping  Tool 

COMBIMAN 

Computerized  Biomechanical  Man  Model 

CPT 

Cockpit  Product  Tool 

CSDP 

Crew-Centered  System  Design  Process 

CSGT 

Crew  Station  Geometry  Tool 

CSS 

Cockpit  System  Specification 

DBMS 

Database  Management  Software 

DOD 

Department  of  Defense 

DT&E 

Developmental  Test  and  Evaluation 

DTIC 

Defense  Technical  Information  Center 

DRD 

Design  Requirements  Document 

DIM 

Design  Traceability  Manager 

DIT 

Design  Trade-off  Tool 

ECM 

Electronic  Counter-Measures 

ECP 

Engineering  Change  Proposal 

EDSIM 

Engineering  Design  Simulator 

FATAT 

Function  Allocation  Trade  Analysis  Tool 

FFAT 

Function  Flow  Analysis  Tool 

GUI 

Graphical  User  Interface 

HFE 

Human  Factors  Engineering 

ICRAT 

Information  and  Control  Requirements  Analysis  Tool 

ILS 

Integrated  Logistics  Support 

IMICS 

Integrated  Master  Information  Control  System 

IPD 

Integrated  Product  Development 

IPT 

Integrated  Product  Team 

IWSM 

Integrated  Weapon  System  Management 

JPATS 

Joint  Primary  Aircraft  Training  System 

LASC 

Lockheed  Aeronautical  Systems  Company 

MCAD 

Mechanical  Computer-Aided  Design 

MDTOOL 

Mission  Decomposition  Tool 

MLTT 

Mechanization  Logic  Tree  Tool 

MOEs 

Measures  of  Effectiveness 

MTM 

Methods-Time  Measurement 

NTIS 

National  Technical  Information  Service 

OT&E 

Operational  Test  and  Evaluation 

PA 

Public  Affairs 

PAWES 

Performance  Assessment  and  Workload  Evaluation  System 

PC 

Personal  Computer 

PIDS 

Prime  Item  Development  Specification 

PIL 

Pilot-in-the-Loop 

PPST 

Program  Planning  and  Scheduling  Tool 

PVI 

Pilot-Vehicle  Interface 

QFD 

Quality  Function  Deployment 

RFP 

Request  for  Proposal 

SAE 

Society  of  Automotive  Engineering 

SEMS 

System  Engineering  Master  Schedule 

SGI 

Silicon  Graphics  Incorporated 

SMEs 

Subject  Matter  Experts 

SON 

Statement  of  Need 

SORD 

System  Operational  Requirements  Document 

SOW 

Statement  of  Work 

SPO 

System  Program  Office 

STPT 

Simulation  Test  Planning  Tool 

SWAS 

Sequitur’s  Workload  Analysis  System 

SWAT 

Subjective  Workload  Assessment  Technique 

TAKE 

Tool  for  Automated  Knowledge  Engineering 

TAR 

Threat  Assessment  Report 

TMT 

Timeline  Management  Tool 

USAF 

United  States  Air  Force 
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VAPS 

Virtual  Avionics  Prototyping  System 

WAT 

Workload  Analysis  Tool 

WBS 

Work  Breakdown  Structure 

WSDP 

Weapon  System  Development  Program 

WSS 

Weapon  System  Specification 

U.S.G.P.O.:  1994-550-057/81088 
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